We at Spilgames are using a lot of python with Vertica.
Over time we developed a Python library which currently features a high speed loader and an extendable base class to write batch importers. Some helper scripts are shipped with as well for instance to easily import csv-like formatted files into Vertica in one command. Depending on used hardware and extra processing, it is easily possible to import 100.000 records / second.
We are now making this library open, under a BSD license. You can find the source on github at https://github.com/spilgames/pyvertica and python package at http://pypi.python.org/pypi/pyvertica with its documentation at http://packages.python.org/pyvertica/ on the pypi repository.
Using it is very easy, for instance:
Code: Select all
from pyvertica.batch import VerticaBatch
batch = VerticaBatch(
dsn='VerticaDWH',
table_name='schema.my_table',
truncate=True,
column_list=['column_1', 'column_2'],
)
row_list = [
['row_1_val_1', 'row_1_val_2'],
['row_2_val_1', 'row_2_val_2'],
...
]
for column_data_list in row_list:
batch.insert_list(column_data_list)
error_bool, error_file_obj = batch.get_errors()
if error_bool:
print error_file_obj.read()
batch.commit()
Guillaume