UnicodeEncodeError: 'ascii' codec can't encode character u'\xe7' in position 17710: ordinal not in range(128)

Question

I'm trying to print a string from an archived web crawl, but when I do I get this error:

print page['html']
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe7' in position 17710: ordinal not in range(128)

When I try print unicode(page['html']) I get:

print unicode(page['html'],errors='ignore')
TypeError: decoding Unicode is not supported

Any idea how I can properly code this string, or at least get it to print? Thanks.

http://farmdev.com/talks/unicode/ – Ignacio Vazquez-Abrams Apr 25 '12 at 19:29 — Ignacio Vazquez-Abrams, Apr 25 '12 at 19:29
This is by far the most common python question. – KurzedMetal Apr 25 '12 at 19:39 — KurzedMetal, Apr 25 '12 at 19:39

score 22 · Accepted Answer · answered Apr 25 '12 at 19:31

You need to encode the unicode you saved to display it, not decode it -- unicode is the unencoded form. You should always specify an encoding, so that your code will be portable. The "usual" pick is utf-8:

print page['html'].encode('utf-8')

If you don't specify an encoding, whether or not it works will depend on what you're printing to -- your editor, OS, terminal program, etc.

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe7' in position 17710: ordinal not in range(128)

1 Answers1

Linked