Pandas parser error when empty CSV column starts showing values?

Asked Aug 15 '19 at 20:03

Active Sep 07 '19 at 21:05

Viewed 88 times

I have a long CSV file (150Mb+) which I am attempting to import, and when read_csv is used, pandas reports this error:

ParserError: Error tokenizing data. C error: Expected 26 fields in line 6100, saw 27

So I checked line 6100, and on the right you can see that one of the columns - which was completely empty up to this point - starts showing values,

My CSV file has 26 columns in the header, with the rightmost column corresponding to one of them, and I have tried different combinations of options, all the way up to

df = pd.read_csv(file_location, header=0, index_col=0, na_values = ["", 0]).fillna(value = 0),

to no avail.

Shouldn't Pandas treat empty cells as N/A's? Why would this cause such a problem?

edited Sep 07 '19 at 21:05

halfer

19,471
17
87
173

asked Aug 15 '19 at 20:03

Coolio2654

1,459
2
16
41

1

nothing concrete but when your first value is `nan` python does some autotyping which leads to error like this one, so maybe that's happening here – Yuca Aug 15 '19 at 20:07
Just add one column name in your header – pythonic833 Aug 15 '19 at 20:09
Have you checked to make sure the values in column 7 don't contain the delimiter you're splitting on? – G. Anderson Aug 15 '19 at 20:09

Pandas parser error when empty CSV column starts showing values?

0 Answers0