Catch exception gets UnboundLocalError

Question

I wrote a crawler to fetch information out of an Q&A website. Since not all the fields are presented in a page all the time, I used multiple try-excepts to handle the situation.

def answerContentExtractor( loginSession, questionLinkQueue , answerContentList) :
    while True:
        URL = questionLinkQueue.get()
        try:
            response   = loginSession.get(URL,timeout = MAX_WAIT_TIME)
            raw_data   = response.text

            #These fields must exist, or something went wrong...
            questionId = re.findall(REGEX,raw_data)[0]
            answerId   = re.findall(REGEX,raw_data)[0]
            title      = re.findall(REGEX,raw_data)[0]

        except requests.exceptions.Timeout ,IndexError:
            print >> sys.stderr, URL + " extraction error..."
            questionLinkQueue.task_done()
            continue

        try:
            questionInfo = re.findall(REGEX,raw_data)[0]
        except IndexError:
            questionInfo = ""

        try:
            answerContent = re.findall(REGEX,raw_data)[0]
        except IndexError:
            answerContent = ""

        result = {
                  'questionId'   : questionId,
                  'answerId'     : answerId,
                  'title'        : title,
                  'questionInfo' : questionInfo,
                  'answerContent': answerContent
                  }

        answerContentList.append(result)
        questionLinkQueue.task_done()

And this code, sometimes, may or may not, gives the following exception during runtime:

UnboundLocalError: local variable 'IndexError' referenced before assignment

The line number indicates the error occurs at the second except IndexError:

Thanks everyone for your suggestions, Would love to give the marks that you deserve, too bad I can only mark one as the correct answer...

Typos, I hand typed it to striped some un-needed lines.. Edited already.. — Paul Liang, Feb 21 '14 at 06:43
Related: [multiple exceptions in one line (except block)](http://stackoverflow.com/questions/6470428/catch-multiple-exceptions-in-one-line-except-block?rq=1) — thefourtheye, Feb 21 '14 at 06:50

Ashwini Chaudhary · Accepted Answer · 2014-02-21T07:02:38.920

I think the problem is this line:

except requests.exceptions.Timeout ,IndexError

This is equivalent to:

except requests.exceptions.Timeout  as IndexError:

So, you're assigning IndexError to the exception caught by requests.exceptions.Timeout. Error can be reproduced by this code:

try:
    true
except NameError, IndexError:
    print IndexError
    #name 'true' is not defined

To catch multiple exceptions use a tuple:

except (requests.exceptions.Timeout, IndexError):

And UnboundLocalError is coming because IndexError is treated as a local variable by your function, so trying to access its value before actual definition will raise UnboundLocalError error.

>>> 'IndexError' in answerContentExtractor.func_code.co_varnames
True

So, if this line is not executed at runtime (requests.exceptions.Timeout ,IndexError) then the IndexError variable used below it will raise the UnboundLocalError. A sample code to reproduce the error:

def func():
    try:
        print
    except NameError, IndexError:
        pass
    try:
        [][1]
    except IndexError:
        pass
func()
#UnboundLocalError: local variable 'IndexError' referenced before assignment

It solved the problem.. Thank you for explaining all the stuffs behind the scene.. I got a much better sense about the code other than just fixing the error.. Thanks again.. — Paul Liang, Feb 21 '14 at 07:10

score 2 · Answer 2 · answered Feb 21 '14 at 06:49

2

When you say

except requests.exceptions.Timeout ,IndexError:

Python will except requests.exceptions.Timeout error and the error object will be IndexError. It should have been something like this

except (requests.exceptions.Timeout ,IndexError) as e:

answered Feb 21 '14 at 06:49

thefourtheye

221,210
51
432
478

score 1 · Answer 3 · answered Feb 21 '14 at 06:49

1

except requests.exceptions.Timeout ,IndexError:

means same as except requests.exceptions.Timeout as IndexError

You should use

except (requests.exceptions.Timeout, IndexError):

instead

answered Feb 21 '14 at 06:49

Kimvais

36,728
16
105
138

Catch exception gets UnboundLocalError

3 Answers3