2

I want to make an post request using curl. The body is a binary file but I also want curl to encode a url-search parameter.

curl --request POST \
     --header 'Content-Type:application/octet-stream' \
     --data-binary "@file.txt" \ # The body part, raw, not encoded
     --data-urlencode "name=john" \ # The url-search-param part, encoded
     "https://example.com"

The problem is that curl treats --data-urlencode as part of the body, what I want is that name=john gets appended to the url part like this: https://example.com?name=john whereby the --data-binary part gets send over as the POST body. (in reality the search-parameter is a string with "invalid" url-characters which need to be encoded)

TLDR: I want to use --data-urlencode as if I'm making a GET request (to append the parameter) and the POST --data-binary to set the actual POST body.

A quick google search gives me the info to use --get / -G but this transforms the request into a GET request what I don't want so there are already a dozen questions about this on SO (but none of them cover my case):

... and many other duplicates of how to just encode the body or url part.

Simon
  • 2,347
  • 1
  • 29
  • 40

2 Answers2

1

You can't.

The --data* options are used to either build the request body, or you can ask curl to put them all as query parameters in the URL (with -G). Those options cannot build both, in a single command line.

My advice would be to build the URL "manually" and keep the --data options to build the request body, mostly because the URL part tends to be easier and smaller.

Daniel Stenberg
  • 49,017
  • 14
  • 131
  • 196
  • Thank you, but what do you mean with "manually", encoding parts of it manually and appending it? I mean, yes, that'd an option and there are already many answers of how to do this but there's no _clean_ way of doing it... – Simon Jun 27 '20 at 11:14
  • @Daniel Stenberg: having `--data-urlencode` and `--data-binary` couldn't be a nice curl enhancement? BTW thanks for building/maintaining curl! – Giorgio Robino May 20 '21 at 08:45
1

curl won't mix post data and query strings for you.

I found a convenient solution for keeping everything in a single bash script without adding additional dependencies. If python, ruby, or node is available, they have url encoding libraries.

Must construct urlencoded query string yourself

A bash only solution for constructing the query string was provided by Chris Down. It is useful gist with an urlencode function that I found suitable. The comments of his gist have modifications for use in other shells.

# From Chris Down https://gist.github.com/cdown/1163649
urlencode() {
    # urlencode <string>
    old_lc_collate=$LC_COLLATE
    LC_COLLATE=C
    local length="${#1}"
    for (( i = 0; i < length; i++ )); do
        local c="${1:$i:1}"
        case $c in
            [a-zA-Z0-9.~_-]) printf '%s' "$c" ;;
            *) printf '%%%02X' "'$c" ;;
        esac
    done
    LC_COLLATE=$old_lc_collate
}

# Construct query string parameter
VAL_1="http://localhost:3030/ds/spek"
ENC_1=$(urlencode "${VAL_1}")
PARAM_1="graph=${ENC_1}"

curl -X PUT --data-binary "@${SPEK_FILE}" \
  --header 'Content-type: application/ld+json' \
  "http://localhost:3030/ds?${PARAM_1}"

# > Content-type: application/ld+json
# > Content-Length: 1215

Using both options appends the data to the body.

Attempting to use both --databinary and --data-encode in a POST ends up with both being appended to the body. Notice the content length difference with the above example.

curl -vvvv -X PUT --data-binary "@${SPEK_FILE}" \
  --header 'Content-type: application/ld+json' \
  --data-urlencode "graph=http://localhost:3030/ds/spek" \
  'http://localhost:3030/ds'

# > Content-type: application/ld+json
# > Content-Length: 1263
GcL
  • 463
  • 6
  • 15