51

After the --dump-header writes a file, how to read those headers back into the next request? I would like to read them from a file because there are a number of them.

I tried standard in: cat headers | curl -v -H - ...

I'm actually using the feature in Firebug to "Copy Request Headers" and then saving those to a file. This appears to be the same format.

jcalfee314
  • 4,120
  • 7
  • 38
  • 71

5 Answers5

77

since curl 7.55.0

Easy:

$ curl -H @header_file $URL

... where the header file is a plain text file with a HTTP header on each line. Like this:

Color: red
Shoesize: 11
Secret: yes
User-Agent: foobar/3000
Name: "Joe Smith"

before curl 7.55.0

curl had no way to "bulk change" headers like that, not even from a file.

Your best approach with an old curl version is probably to instead write a shell script that gathers all the headers from the file and use them, like:

#!/bin/sh
while read line; do
  args="$args -H '$line'";
done
curl $args $URL

Invoke the script like this:

$ sh script.sh < header_file
Daniel Stenberg
  • 49,017
  • 14
  • 131
  • 196
  • can you add an example of the header file? i'd like to see the format. how to put in strings, null, bool, int, float; where to use quotes. i just keep getting this as a search result so I think it would help people – kirill_igum Aug 13 '20 at 23:50
  • 1
    Added. But really exactly how each header is supposed to look like or contain depends entirely on that specific header... – Daniel Stenberg Aug 14 '20 at 15:02
60

how about this:

curl -v -H "$(cat headers.txt)" yourhost.com

where headers.txt looks like

Header1: bla
Header2: blupp

works in BASH.

Cpt. Senkfuss
  • 1,318
  • 3
  • 12
  • 19
  • Works in curl 7.22.0. I added a space so it would be 'bla bla' and I could not trip it up. This command was esier to test with `curl -v -H "$(cat headers.txt)" google.com` .. Lets see if anyone else comments to make sure this is a safe way to do it. It is not documented. – jcalfee314 Aug 06 '14 at 14:29
  • I explicitly stripped the `-v` before since I thought it's noise that's not adding to a correct answer, but you're right, it is easier. so I added it back – Cpt. Senkfuss Aug 08 '14 at 09:56
  • 3
    But it works more or less by coincidence since it feeds it in as a single header with embedded newlines. IMHO, A cleaner version would have a -H on each line in the headers.txt file too and leave it out from the command line. – Daniel Stenberg Mar 10 '15 at 14:44
  • I like this hack for its elegance, but there's one tiny problem: because headers are not explicitly listed individually, they don't override any default headers sent by curl. Result: duplicate headers. E.g. if you have a custom content-type header, combined with -d, you now send two, possibly conflicting, content-type headers. – hraban Jun 14 '18 at 14:19
  • 3
    This will work, HOWEVER, keep in mind that in a process listing (such as `ps aux`), the expanded variable may be seen by other users. *SO*, if you are providing credentials or auth-token in the header, this approach will _expose the credentials_ to other users - which would be unacceptable. – Yoshiya Jan 23 '20 at 13:55
28

Starting with curl 7.55.0 it can now read headers from a file:

curl -H @filename

It's that easy now.

Dmitrii Sutiagin
  • 571
  • 5
  • 8
  • 1
    Ok, but what's the syntax? 'header':'info' "header":"info" header:info or some other combination? I can't seem to find the right combination. – Chris Troutner Jul 23 '18 at 18:26
  • 1
    Chris, the syntax is the same as in CLI - "name: value", no quotes, 1 header per line. You can test what is sent by using -v flag: `curl -v -H @test.txt test.com` to see how it's actually sent and compare with `-H 'x: y' -H 'a: b'` - you should see identical output from curl as long as your `test.txt` contains 2 lines: `x: y` and `a: b`. – Dmitrii Sutiagin Jul 24 '18 at 22:11
12

As answered by @dmitry-sutyagin, if your curl is at least version 7.55.0 you can use the @ notation to read headers from a file:

curl -H @headerfile.txt https://www.google.com/  # requires curl 7.55.0

If your curl is NOT 7.55.0 or newer, there's a useful hack:

  • Use the option -K/--config <config file>, and put several -H/--header <header> lines in the text file.

For instance:

  1. curl --dump-header foo.txt https://www.google.com/
  2. If necessary, dos2unix foo.txt

  3. Convert the file to -H 'header' lines, manually or with a script:

    cat foo.txt |
  awk '$1 == "Set-Cookie:"' |
  perl -ne "chomp; next if /^\\s*\$/; if (/'/) { warn; next } print \"-H '\$_'\\n\";" |
  tee headerfile.txt

    This might output something like:

    -H 'Set-Cookie: 1P_JAR=2018-02-13-08; [...]'
-H 'Set-Cookie: NID=123=n7vY1W8IDElvf [...]'

  4. curl --config headerfile.txt https://www.google.com/

traal
  • 421
  • 3
  • 7
  • 2
    Using the default curl 7.54.0 on macOS Mojave, I had to use double quotes instead of single quotes in the config file, otherwise I got warnings about unescaped whitespace and the headers weren't set at all – CupawnTae Jun 05 '19 at 13:47
5
curl $(xargs -a headers.txt printf "-H '%s'") example.org
user2683246
  • 3,169
  • 27
  • 30