30

I'm calling some JSON and parsing relevant data as CSV. I cannot figure out how to fill in the intermediate JSON dict file with default keys, as many are unpopulated. The result is a KeyError as I attempt to parse the content into a CSV.

I'm now receiving a 'NoneType' error for (manufacturer):

import urllib2, json, csv, sys, os, codecs, re

from collections import defaultdict

output = 'bb.csv'

csv_writer = csv.writer(open(output, 'w'))

header = ['sku', 'name', 'description', 'image', 'manufacturer', 'upc', 'department', 'class', 'subclass']

csv_writer.writerow(header)

i=1

while i<101:
    print i

    bb_url = urllib2.Request("http://api.remix.bestbuy.com/v1/products(sku=*)?show=sku,name,description,image,manufacturer,upc,department,class,subclass&format=json&sort=sku.asc&page=" + str(i) + "&pageSize=100&apiKey=*****************")
    bb_json = json.load(urllib2.urlopen(bb_url))

    print bb_json

    for product in bb_json['products']:
        row = []

        row.append(product['sku'])
        if product['name']:
            row.append(str((product['name']).encode('utf-8')))
        else:
            row.append("")
        row.append(str(product.get('description',"")))
        row.append(str(product['image'])+ " ")
        if product['name']:
            row.append(str(product.get('manufacturer',"").encode('utf-8')))
        else:
            row.append("")
        row.append(str(product.get('upc','').encode('utf-8')))
        row.append(str((product['department']).encode('utf-8')))
        row.append(str((product['class']).encode('utf-8')))
        row.append(str((product['subclass']).encode('utf-8')))

        csv_writer.writerow(row)

    i = i+1
jonrsharpe
  • 107,083
  • 22
  • 201
  • 376
Kenfucious
  • 369
  • 1
  • 5
  • 9

4 Answers4

71

You can use your_dict.get(key, "default value") instead of directly referencing a key.

Noam Manos
  • 12,732
  • 2
  • 76
  • 75
Stack of Pancakes
  • 1,771
  • 16
  • 23
  • Thanks for assisting. That line would be inserted before the for loop, or iteratively within? – Kenfucious Jul 17 '14 at 21:54
  • 2
    Whenever you'd normally access a key. So when you go to get the product's name instead of `row.append(product['name'])` you would use `row.append(product.get('name', 'value if name key doesn't exist'))` – Stack of Pancakes Jul 17 '14 at 21:57
  • Thanks, that seemed to work!! One final point: I'm now getting a nonetype error for one of the fields (manufacturer). The if clause for 'name' works well, but a similar syntax yields the dreaded KeyError once again. – Kenfucious Jul 17 '14 at 22:36
  • 2
    Don't use `None` for the default value. In your code you're setting it to `""` if the value doesn't exist. So try `row.append(product.get('name', ''))` Which will append the value for `name` if it exists, or an empty string if it doesn't. – Yep_It's_Me Jul 17 '14 at 22:48
  • Notice that "dict" shouldn't be the name of your dictionary, since "dict" is a reserved built-in symbol in Python – Noam Manos Jan 29 '16 at 20:22
9

Don't use the "default" argument name. For example, if we want 1.0 as default value,

rank = dict.get(key, 1.0)

For more details: TypeError: get() takes no keyword arguments

Community
  • 1
  • 1
Serendipity
  • 2,086
  • 22
  • 32
8

If you can't define a default value and want to do something else (or just omit the entry):

if key in dict:
    rank = dict[key]
else:
    # do something or just skip the else block entirely
Alceu Rodrigues de Freitas Jun
  • 91
  • 1
  • 3
2

You could use syntax like this: product.get("your field", "default value")

Eugen
  • 2,190
  • 3
  • 28
  • 42