90

I'm trying to print just the verbose sections of a cURL request (which are sent to stderr) from the bash shell.

But when I redirect stdout like this:

curl -v http://somehost/somepage > /dev/null

Some sort of results table appears in the middle of the output to stderr:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

Followed by this near the end:

{ [data not shown]
118   592    0   592    0     0  15714      0 --:--:-- --:--:-- --:--:-- 25739

Which makes the response headers less readable.

I don't see this text when not redirecting.

Another way to see the effects:

Table doesn't appear:

curl -v http://somehost/somepage 2>&1

Table appears:

curl -v http://somehost/somepage 2>&1 | cat

  1. How come this shows up only with certain types of redirects?

  2. What's the neatest way to suppress it?

ZygD
  • 2,497
Ian Mackinnon
  • 5,686
  • 5
  • 30
  • 34

8 Answers8

74

Try this:

curl -vs -o /dev/null http://somehost/somepage 2>&1

That will suppress the progress meter, send stdout to /dev/null and redirect stderr (the -v output) to stdout.

Dennis Williamson
  • 108,309
  • 9
    @IanMackinnon Note that with -s but without -v you will not see errors such as failure to connect. For that you should also add -S (or --show-error) as in mhoydis's answer. – Artyom Apr 14 '14 at 09:47
  • but why does the progress bar only appear in the first place when redirecting? I ran into this same issue when piping the output of curl to jq. No progress bar without piping to jq, then when piping to jq I have to go back and add -s. – sixty4bit Jul 29 '19 at 17:54
  • @sixty4bit: That's a design choice the developers made. The program can detect when its STDOUT isn't a tty. When output is not being piped, you don't want progress information to be interspersed with normal output, which you can see and have some idea of progress. When output is redirected or piped, you can't see it so you have no gauge for progress - unless the progress bar is turned on. – Dennis Williamson Jul 29 '19 at 20:18
32
curl --fail --silent --show-error http://www.example.com/ > /dev/null

This will suppress the status dialog, but will otherwise output errors to STDERR.

user@host:~# curl http://www.yahoo.com > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  254k    0  254k    0     0   403k      0 --:--:-- --:--:-- --:--:--  424k

The above outputs the status table when redirecting.

user@host:~# curl --fail --silent --show-error http://www.yahoo.com > /dev/null

The above suppresses the status table when redirecting, but errors will still go to STDERR.

user@host:~# curl --fail --silent --show-error http://www.errorexample.com > /dev/null
curl: (6) Couldn't resolve host 'www.errorexample.com'

The above is an example of an error to STDERR.

user@host:~# curl -v --fail --silent --show-error http://www.errorexample.com > ~/output.txt 2>&1
user@host:~# cat ~/output.txt 
* getaddrinfo(3) failed for www.errorexample.com:80
* Couldn't resolve host 'www.errorexample.com'
* Closing connection #0
curl: (6) Couldn't resolve host 'www.errorexample.com'

Just add 2>&1 to the end to redirect STDERR to STDOUT (in this case, to a file).

user620965
  • 103
mhoydis
  • 421
9

According to man curl:

-s, --silent : Silent or quiet mode. Don't show progress meter or error messages. Makes Curl mute.

Example usage:

curl -s 'http://www.google.com'

or if you want to capture the HTTP-BODY into a variable in bash

BODY=$( curl -s 'http://www.google.com' )
echo $BODY

You can use -s or --silent interchangeably.

Kamil Maciorowski
  • 74,068
Venkatt Guhesan
  • 91
  • 1
  • 2
6

Modern versions of curl now have the option --no-progress-meter which disables only the progress meter you are referencing.

Michael Hampton
  • 13,885
5

With reference to question 1 (how cURL knows to only display the table when output is redirected), I didn't realise a program could tell its outputs were being directed, but it seems on POSIX systems there is a function isatty which reports whether or not a file descriptor refers to a terminal.

Ian Mackinnon
  • 5,686
  • 5
  • 30
  • 34
  • 3
    Here's a Bash snippet: [[ -p /dev/stdout ]] && echo "stdout is to a pipe"; [[ -t 1 ]] && echo "output to terminal"; [[ ! -t 1 && ! -p /dev/stdout ]] && echo "output redirected" – Dennis Williamson Aug 07 '10 at 20:41
2

1) How come this shows up only with certain types of redirects?

from the curl man page

If you want a progress meter for HTTP POST or PUT requests, you need to redirect the response output to a file, using shell redirect (>), -o [file] or similar.

curl must use isatty to determine the redirect and prints the progress meter when redirected to a file or shell pipe.

2) What's the neatest way to suppress it?

from the curl man page

-s, --silent

Silent or quiet mode. Don't show progress meter or error messages. Makes Curl mute. It will still output the data you ask for, potentially even to the terminal/stdout unless you redirect it.

Community
  • 1
Wyrmwood
  • 216
1

To put real error messages somwhere, you should write strerr into a log file. Something like that:

curl  "http://domain.name/process" --stderr /var/log/curl_err.log > /dev/null
user1065951
  • 11
0

Being behind a proxy I use a command like this.

date -s "$(curl --proxy http://PROXY:8080 -s http://google.com --head -s |grep Date|sed 's/Date: //g')"

VeggieVampire
  • 27
  • 2