20

I have the following code:

    tries = 10
    for n in range(tries):
        try:
            ....
            responsedata = requests.get(url, data=data, headers=self.hed, verify=False)
            responsedata.raise_for_status()
            ..
            if .... : 
                break   #exit loop condition

        except (ChunkedEncodingError, requests.exceptions.HTTPError) as e:
            print ("page #{0} run #{1} failed. Returned status code {2}. Msg: {3}. Retry.".format(page, n, responsedata.status_code, sys.exc_info()[0]))
            if n == tries - 1:
               raise e  # exit the process

The prints I see are:

page #53 run #0 failed. Returned status code 502. Msg: <class 'requests.exceptions.HTTPError'>. Retry.
page #1 run #1 failed. Returned status code 500. Msg: <class 'requests.exceptions.ChunkedEncodingError'>. Retry.

While this is Ok it doesn't give me actual information about the problem. The message just tell me the exception title.

If I print the: responsedata.text when exception happens I see:

 Returned status code 502. Message is: ...
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>
<title>502 - Web server received an invalid response while acting as a gateway or proxy server.</title>
<style type="text/css">
<!--
body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}
fieldset{padding:0 15px 10px 15px;}
h1{font-size:2.4em;margin:0;color:#FFF;}
h2{font-size:1.7em;margin:0;color:#CC0000;}
h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;}
#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS", Verdana, sans-serif;color:#FFF;
background-color:#555555;}
#content{margin:0 0 0 2%;position:relative;}
.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;position:relative;}
-->
</style>
</head>
<body>
...

This is a giant message most of it is garbage but it also says: 502 - Web server received an invalid response while acting as a gateway or proxy server. can I access this message and also print it to my log?

Vitor Baptista
  • 1,721
  • 1
  • 18
  • 26
Programmer120
  • 1,952
  • 7
  • 24
  • 45
  • @PedroLobito I'm aware of that but the API can raise more types or errors, some can be unique. I'm asking how to access the message using request package – Programmer120 Sep 04 '18 at 13:32

2 Answers2

29

You can access the response's status code using responsedata.status_code and its textual description via responsedata.reason (see more in http://docs.python-requests.org/en/master/api/)

Vitor Baptista
  • 1,721
  • 1
  • 18
  • 26
  • Nice but it contains a diffrent message. For example it says: `Bad Gateway` instead of `Web server received an invalid response while acting as a gateway or proxy server.` – Programmer120 Sep 04 '18 at 13:47
  • 1
    That message is in the response's title, added by the webserver. In other words, it's specific to this URL you're accessing, not to any webserver (as my answer). If you want to extract it, you'll need to parse the HTML and extract its `title` tag. You can see how on https://stackoverflow.com/questions/26812470/how-to-get-page-title-in-requests. – Vitor Baptista Sep 04 '18 at 13:53
  • 1
    Adding to my comment, it seems this message is just the default message IIS adds for status code 500. It doesn't add more information, it's just more verbose. Considering that extracting it would involve parsing the HTML, I'd suggest just logging the HTTP Error code, as it has the same information. If you want to extract it anyway, the solution is linked in my previous comment. – Vitor Baptista Sep 04 '18 at 13:58
1

If your endpoint returns a detailed, application-specific error message in the response body, you can make use of the fact that a requests HTTPError retains a reference to the Response that caused it to be raised:

from requests.exceptions import HTTPError

try:
    # Some code that makes requests
except HTTPError as e:
    print(e.response.text)
user1898153
  • 336
  • 3
  • 10