64

Hi: I am trying to use the Pandas DataFrame.to_csv method to save a dataframe to a csv file:

filename = './dir/name.csv'

df.to_csv(filename)

However I am getting the error:

IOError: [Errno 2] No such file or directory: './dir/name.csv'

Shouldn't the to_csv method be able to create the file if it doesn't exist? This is what I am intending for it to do.

kgangadhar
  • 4,369
  • 4
  • 30
  • 50
LoLa
  • 1,050
  • 1
  • 9
  • 17

3 Answers3

107

to_csv does create the file if it doesn't exist as you said, but it does not create directories that don't exist. Ensure that the subdirectory you are trying to save your file within has been created first.

I often do something like this in my work:

import os

outname = 'name.csv'

outdir = './dir'
if not os.path.exists(outdir):
    os.mkdir(outdir)

fullname = os.path.join(outdir, outname)    

df.to_csv(fullname)

This can easily be wrapped up in a function if you need to do this frequently.

qRTPCR
  • 1,346
  • 1
  • 11
  • 13
  • 12
    `os.mkdir` creates only one immediate directory. If there are several levels of directories in the path that don't exist, use `os.makedirs`. However note that it throws an `OSError` if the leaf exists – mz8i Aug 24 '18 at 09:13
  • 5
    @mz8i If an existing leaf directory is acceptable, `os.makedirs` can be called with `exist_ok=True` to avoid the error. – stiaan Mar 30 '20 at 15:05
50

Here is an alternative way to do this using the excellent standard library pathlib module, which generally makes things neater.

As explained elsewhere, to_csv will create the file if it doesn't exist, but won't create any non-existent directories in the path to the file, so you need to first ensure that these exist.

from pathlib import Path

output_file = 'my_file.csv'
output_dir = Path('long_path/to/my_dir')

output_dir.mkdir(parents=True, exist_ok=True)

df.to_csv(output_dir / output_file)  # can join path elements with / operator

Setting parents=True will also create any necessary parent directories, and exist_ok=True means it won't raise an error if the directory already exists, so you don't have to explicitly check that separately.

Tim
  • 1,591
  • 9
  • 17
  • 1
    This was what is wanted, I also thought of creating directories recursively using the above implementation. But this is better and much simpler. – Janith Feb 15 '19 at 05:56
  • 5
    `pathlib` is such a dream! :D – Tim Feb 15 '19 at 21:35
  • 6
    @JKK You can do the exact same thing with `os.makedirs('long_path/to/my_dir', exist_ok=True)` - has the exact same behavior. However, nothing wrong with the object oriented `pathlib` approach if that's your cup of tea! – qRTPCR May 29 '19 at 20:52
  • 1
    If I'm completely honest, this was partly just an opportunity to introduce people to `pathlib` ...and yes, speaking as a Brit, it is very much my cup of tea! ☕ – Tim Jun 01 '19 at 16:12
  • This should be the accepted answer. Much cleaner and safer solution. – Crispy Holiday Nov 08 '21 at 15:23
2

I had this error when I accidentally added file:// at the begging of the save path. Since search brought me here, might be also helpful to someone.

Koby
  • 130
  • 2
  • 13