2

I'm reading a CSV file into a namedtuple as so:

import csv
from collections import namedtuple

#So we can handle bad CSV files gracefully
def unfussy_reader(reader):
    while True:
        try:
            yield next(reader.lower())

        # This is a bad row that has an error in it (csv.Error)
        # Alternately it may be a line that doesn't map to the structure we've been given (TypeError)
        except (csv.Error, TypeError):
            pass

        continue

# Create the CSV reader object
csv_reader = csv.reader(file_stream, delimiter=' ', quotechar='"', escapechar='^')

# Set up the named tuple
csvline = namedtuple('csv_line', 'field1, field2, field3')

# Create the named tuple mapping object
map_to_tuple = map(csvline._make, csv_reader)

for line in unfussy_reader(map_to_tuple):
    # do stuff

This works well, but my problem is - I want all of the content of the CSV to be read in lower-case. As per this question, a simple lambda would do it: map(lambda x:x.lower(),["A","B","C"]) but I can't find anywhere to put it before the data ends up in the tuple (and thus unchaneable).

Is there a way to do this within this structure (Python 3.5)?

Community
  • 1
  • 1
GIS-Jonathan
  • 3,697
  • 7
  • 29
  • 43
  • 1
    Add a line before `csv_reader = ...` that does: `file_stream = (line.lower() for line in file_stream)`? – bbayles Jan 03 '16 at 16:58
  • @bbayles - that worked, although I had to assign it to something and then change the `csv_reader = ` line to point at that rather than `file_stream`. Post it as an answer and I'll accept. Thanks – GIS-Jonathan Jan 03 '16 at 17:09

2 Answers2

4

You can apply the lower transform to the stream before you create a CSV reader for it.

lower_stream = (line.lower() for line in file_stream)
csv_reader = csv.reader(lower_stream, delimiter=' ', quotechar='"', escapechar='^')

The parentheses around the lower_stream assignment target designate a generator expression. It will not use up file_stream and will not pull all of file_stream into memory.

bbayles
  • 4,094
  • 1
  • 25
  • 34
0

How about this:

csv_reader = csv.reader(map(lambda line:line.lower(),file_stream), delimiter=' ', quotechar='"', escapechar='^')
Iron Fist
  • 10,237
  • 2
  • 17
  • 32