Hello people.
I'm working in a tool, I'm using mysql binlog to replicate into vertica in real time. I'm using python between mysql / vertica, but the INSERT statement is very slow, there are other way to accelerate the INSERT?
thanks in advance
INSERT very slow
Moderator: NorbertKrupa
- JimKnicely
- Site Admin
- Posts: 1825
- Joined: Sat Jan 21, 2012 4:58 am
- Contact:
Re: INSERT very slow
Have you tried adding a DIRECT hint? You'll write directly to the ROS skipping the WOS.
For example:
INSERT /*+ DIRECT */ INTO mytab ....
For example:
INSERT /*+ DIRECT */ INTO mytab ....
Jim Knicely
Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.
Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.
Re: INSERT very slow
We need more details. Is this a large INSERT..SELECT that is slow? Or are you inserting the data row by row with individual INSERT statements? If the latter, you should batch them using the faster COPY bulk load. Also if it’s individual INSERT statements, don’t add the direct hint, because then you’re likely to get “too many ROS containers”. Also a direct load is slower than a load to WOS since direct loads need to sort the data.
—Sharon
—Sharon
Sharon Cutter
Vertica Consultant, Zazz Technologies LLC
Vertica Consultant, Zazz Technologies LLC
Re: INSERT very slow
Yes, I'm inserting row by row with individual INSERT statement.
I receive the data from binlog (mysql) and then I insert into vertica, I'm using ODBC / python2.7
Could you help me?
Thanks
I receive the data from binlog (mysql) and then I insert into vertica, I'm using ODBC / python2.7
Could you help me?
Thanks
Re: INSERT very slow
Hi!
[DELETED]
[DELETED]
Last edited by id10t on Wed May 06, 2015 5:32 pm, edited 1 time in total.
Re: INSERT very slow
Hi Skwa,
I was not sure on the Pyvertica usage last year, but it seems there are many new features with this driver now.
Thanks for pointing us to this.
Surely going to try this.
I was not sure on the Pyvertica usage last year, but it seems there are many new features with this driver now.
Thanks for pointing us to this.
Surely going to try this.
Re: INSERT very slow
I'll try to test your example.
Thanks so much!!
Thanks so much!!