How can I make url.el work with the If-None-Match header?

Question

Take the following form constructed according to :

(let ((break-url-package nil))
  (let ((url-request-method "GET")
        (url-request-extra-headers
         `(("Content-Type" . "application/json")
           ,@(when break-url-package
               '(("If-None-Match" . "\"00d61dfb6398b3994d4c7110b0eb1b9f\""))))))
    (switch-to-buffer-other-window
     (url-retrieve-synchronously
      "https://api.github.com/users/octocat"))))

which gives the following response:

HTTP/1.1 200 OK
Server: GitHub.com
Date: Fri, 19 May 2017 02:29:55 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Status: 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 38
X-RateLimit-Reset: 1495163787
Cache-Control: public, max-age=60, s-maxage=60
Vary: Accept
ETag: W/"00d61dfb6398b3994d4c7110b0eb1b9f"
Last-Modified: Wed, 17 May 2017 21:18:07 GMT
X-GitHub-Media-Type: github.v3; format=json
Access-Control-Expose-Headers: ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
Access-Control-Allow-Origin: *
Content-Security-Policy: default-src 'none'
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-XSS-Protection: 1; mode=block
Vary: Accept-Encoding
X-Served-By: e14705a23c085afeff5e104b1fc3922a
Content-Encoding: gzip
X-GitHub-Request-Id: E541:15878:199148E:1E81AEB:591E58A1

{"login":"octocat","id":583231,...}

If I let (break-url-package t), it's a whole different story:

Contacting host: api.github.com:443
error in process filter: insert-file-contents-literally: Opening input file: No such file or directory, /Users/sean/.emacs.d/url/cache/sean/https/com/github/api/84a3031e21fe0081db5ec1586ae7cbb6
error in process filter: Opening input file: No such file or directory, /Users/sean/.emacs.d/url/cache/sean/https/com/github/api/84a3031e21fe0081db5ec1586ae7cbb6

After those messages print, Emacs hangs (until C-g).

No other headers appear to behave this way.

After some more investigation, I've found this is the exact request I'm making to the server according to (url-http-create-request):

GET /users/octocat HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: api.github.com
Accept-encoding: gzip
Accept: */*
User-Agent: URL/Emacs
Cookie: HttpOnly=nil; logged_in=no
Content-Type: application/json
If-None-Match: "d093767a41c43cbc64dcc3bda1b5880f"

where

curl -i https://api.github.com/users/octocat \
  -H 'MIME-Version: 1.0' \
  -H 'Connection: keep-alive' \
  -H 'Extension: Security/Digest Security/SSL' \
  -H 'Host: api.github.com' \
  -H 'Accept-encoding: gzip' \
  -H 'Accept: */*' \
  -H 'User-Agent: URL/Emacs' \
  -H 'Cookie: HttpOnly=nil; logged_in=no' \
  -H 'Content-Type: application/json' \
  -H 'If-None-Match: "d093767a41c43cbc64dcc3bda1b5880f"'

responds with

HTTP/1.1 304 Not Modified
Server: GitHub.com
Date: Fri, 19 May 2017 11:54:11 GMT
Status: 304 Not Modified
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 56
X-RateLimit-Reset: 1495197406
Cache-Control: public, max-age=60, s-maxage=60
Vary: Accept
ETag: "d093767a41c43cbc64dcc3bda1b5880f"
Last-Modified: Wed, 17 May 2017 21:18:07 GMT
etc... just headers

score 1 · Answer 1 · answered May 19 '17 at 12:28

As the error implies, there was no such file /.../https/com/github/api/.... This is needed by

(url-cache-extract (url-cache-create-filename (url-view-url t)))

in url-http.el. Once I realized that url already had support for HTTP-304 caching, I discovered url-automatic-caching which seems to do the trick. However, now it's not setting url-http-end-of-headers correctly, but that's a different question :-)

Working example:

(let ((url-request-method "GET")
      (url-automatic-caching t))
  (switch-to-buffer-other-window
   (url-retrieve-synchronously
    "https://api.github.com/users/octocat")))

How can I make url.el work with the If-None-Match header?

1 Answers1

Linked