2

I am running a python script using pandas.read_csv to import a csv file. Pandas provides console warnings when it doesn't see what it expects such as:

Skipping line 163: Expected 41 fields in line 163, saw 42

How can I log this to a text file?

If I run the script from a command line, python > logfile.txt only the output of print shows up in the file, not the warnings.

Arkistarvh Kltzuonstev
  • 6,572
  • 7
  • 24
  • 47
Michael C
  • 185
  • 5

2 Answers2

4

Use stderr

In bash, there are 3 default streams:

STDOUT: Redirected by > or 1>, this is standard output from a program.

STDERR: Redirected by 2>, this is diagnostic output from a program.

STDIN: Input from console, use < to input it. 


./prog 2> errorlog.txt >logfile.txt

If you want to redirect ALL output to STDOUT, use:

./prog 2>&1

Here is some more information: I/O Redirection

blackbrandt
  • 1,768
  • 14
  • 29
0

This line worked for me

python script.py >>script.out 2>&1

which appends STDOUT (1) to script.out file and redirects STDERR (2) to the same file (&1)

Note that output is appended, so you may want to call > script.out to reset the file beforehand, as in

> script.out; python script.py >> script.out 2>&1

Sources

Franco Marchesoni
  • 5
  • 5