In ruby on rails application how can i upload 3 GB csv file to the pg database on server

Question

I want to upload 3 GB csv file to the pg database on server.

I need to finish this operation in 4 to 5 hours.

Please show me the way to upload data to server from csv file.

you need http://www.postgresql.org/docs/9.4/static/sql-copy.html — Vivek S., Sep 07 '15 at 09:51
Worth mentioning that you can use `scp` to get the file onto your server in the first place. Don't use Git. — Max Williams, Sep 07 '15 at 10:06
See also http://stackoverflow.com/q/758945/398670, http://stackoverflow.com/q/12206600/398670 — Craig Ringer, Sep 07 '15 at 10:13

score 0 · Answer 1 · edited May 23 '17 at 11:58

0

If you can take the DB offline, copy the file to the server with (e.g.) scp, then use pg_bulkload.

Otherwise split it into chunks and use psql's \copy to load the chunks in parallel, after dropping indexes etc.

1 Answers1