Export mysql query to amazon s3 bucket

Question

Based on the accepted answer here, I am able to export the results of mysql query to a csv file on my Amazon EC2 instance using the following:

mysql -user -pass -e "SELECT * FROM table" > /data.csv

However, as the file exported is large, I want to export an Amazon s3-bucket (s3:\\mybucket) which is accessible from my EC2 instance

I tried:

mysql -user -pass -e "SELECT * FROM table" > s3:\\mybucket\data.csv

But it doesn't export the file.

You will have to do it in 2 steps, first safe the file to your instance and then send it to S3. — jarnohenneman, Dec 04 '17 at 10:55
@jarnohenneman but the output file is bigger than the instance disk space — mallet, Dec 04 '17 at 10:57

score 1 · Accepted Answer · answered Dec 04 '17 at 20:05

If you want to use the mysql command line program, then you have two choices:

Increase the size of your instance's storage so that the file can be created. Then copy the file to S3
Create a separate program or script that reads from Standard Input and writes to S3.

Another solution would be to create a simple program that processes your SELECT and directly writes to S3. There are lots of examples of this on the Internet in Python, Java, etc.

Export mysql query to amazon s3 bucket

1 Answers1