Why isn't url-http-end-of-headers set correctly when using url-automatic-caching?

Question

This is a follow-up question to How can I make url.el work with the If-None-Match header?

Evaluate the following form:

(let ((url-request-method "GET")
      (url-automatic-caching t))
  (with-current-buffer
      (url-retrieve-synchronously
       "https://api.github.com/users/octocat")
    (message "'%c' (%S)"
             (char-after url-http-end-of-headers)
             url-http-end-of-headers)))

It thinks url-http-end-of-headers is position 1 in the following buffer:

HTTP/1.1 200 OK
Server: GitHub.com
Date: Fri, 19 May 2017 12:18:32 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Status: 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 55
X-RateLimit-Reset: 1495197406
Cache-Control: public, max-age=60, s-maxage=60
Vary: Accept
ETag: W/"d093767a41c43cbc64dcc3bda1b5880f"
Last-Modified: Wed, 17 May 2017 21:18:07 GMT
X-GitHub-Media-Type: github.v3; format=json
Access-Control-Expose-Headers: ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
Access-Control-Allow-Origin: *
Content-Security-Policy: default-src 'none'
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-XSS-Protection: 1; mode=block
Vary: Accept-Encoding
X-Served-By: 075bb2f6b7031ca3c0e69edb17939fae
Content-Encoding: gzip
X-GitHub-Request-Id: E86D:15875:2064CAA:268A8E5:591EE291

{"login":"octocat","id":583231,...}

Couldn't reproduce in Emacs 26.2, I got (#<marker at 1034 in *http api.github.com:443*>), which is the position of the newline at the end of the headers. — npostavs, May 04 '19 at 16:34
@npostavs that is consistent with what we've seen on #81. I wonder then if this question is no longer relevant. — Sean Allred, May 04 '19 at 21:03
I don't think #81 (aka Emacs Bug#34341) is related, in that case (gnutls v3.6+) the response buffer is entirely empty. — npostavs, May 05 '19 at 02:39

Sean Allred · Answer 1 · 2017-05-31T00:41:58.477

1

This seems to be a bug in emacs at url-cache-extract with its use of erase-buffer and wiping out that marker. More details when I'm not mobile. I'll be submitting a bug report :-)

This seems to be a bug in either url-http-parse-headers or url-cache-extract. Before url-http-parse-headers calls url-cache-extract, url-http-end-of-headers is set correctly. However, when url-cache-extract calls erase-buffer to replace the response buffer (which has header HTTP-304), url-http-end-of-headers is blown away (rightly so), but not reset.

Redefining url-cache-extract to find the end of the headers and set the marker appropriately should fix the immediate issue, but I'll be reporting this bug to Emacs' bug tracker since I'm pretty sure this isn't all that needs to happen in this case.

edited May 31 '17 at 00:41

answered May 19 '17 at 15:02

Sean Allred

6,921
18
86

Any updates on this? – imalison Mar 07 '19 at 04:35
@imalison no updates, but you might be interested in the discussion at https://github.com/magit/ghub/issues/81 which describes a similar problem (though I doubt it's exactly the same). It may need to be brought up on emacs-devel again, but I regret that I don't have the time to do this right now or in the near future, so I'd recommend you do so if you need a fix. – Sean Allred Mar 07 '19 at 05:08

Why isn't url-http-end-of-headers set correctly when using url-automatic-caching?

1 Answers1