Hi all,
We are using vertica 7.2.1 and we have 50 odd tables in a schema (total data size ~5 TB). What is the best approach you would suggest to move this data into S3?
Exporting Vertica tables to S3
Moderator: NorbertKrupa
- JimKnicely
- Site Admin
- Posts: 1825
- Joined: Sat Jan 21, 2012 4:58 am
- Contact:
Re: Exporting Vertica tables to S3
Have you check out "Exporting to Amazon S3 using the Vertica Library for Amazon Web Services"?
https://my.vertica.com/docs/8.0.x/HTML/ ... ibrary.htm
https://my.vertica.com/docs/8.0.x/HTML/ ... ibrary.htm
Jim Knicely
Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.
Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.
Re: Exporting Vertica tables to S3
Yes I did check them out. But we are on Vertica 7.2.1. Other than upgrading and using awslib, is there any better way?
- JimKnicely
- Site Admin
- Posts: 1825
- Joined: Sat Jan 21, 2012 4:58 am
- Contact:
Re: Exporting Vertica tables to S3
Hi,
Maybe you can mount an S3 Bucket per the the instructions in the article "How to Mount S3 Bucket on CentOS/RHEL and Ubuntu using S3FS"?
https://tecadmin.net/mount-s3-bucket-ce ... sing-s3fs/
Once you have the S3 Bucket mounted, you should be able to export data out of Vertica onto it.
vsql -c "select * from some_table;" -o /s3mnt/some_table.txt
Maybe you can mount an S3 Bucket per the the instructions in the article "How to Mount S3 Bucket on CentOS/RHEL and Ubuntu using S3FS"?
https://tecadmin.net/mount-s3-bucket-ce ... sing-s3fs/
Once you have the S3 Bucket mounted, you should be able to export data out of Vertica onto it.
vsql -c "select * from some_table;" -o /s3mnt/some_table.txt
Jim Knicely
Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.
Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.