how can I parse json with a single line python command?

Question

I would like to use python to parse JSON in batch scripts, for example:

HOSTNAME=$(curl -s "$HOST" | python ?)

Where the JSON output from curl looks like:

'{"hostname":"test","domainname":"example.com"}'

How can I do this with a single line python command?

Much easier to use something like `jq` from the command line, which is dedicated to parsing JSON and so has a cleaner design: `HOSTNAME=$(curl -s $HOST | jq -r '.hostname')`. — chepner, Jul 20 '16 at 17:39
Down voters, please state why you are down voting. It's difficult to improve questions if you don't know what's wrong with them. — Chris Snow, Jul 20 '16 at 17:41
@chepner true, but I don't have and can't install jq on the host in question, but I do have python and the json module. — Chris Snow, Jul 20 '16 at 17:43
For what purpose do you require the Python code to be "a single line"? — Jongware, Jul 23 '16 at 10:01
It's much easier to work with single line commands in a batch script. Multi line Python statements that require Python indentation are difficult to integrate into scripts using pipes and redirects without having to put the Python statements into a separate file as a Python script. — Chris Snow, Jul 23 '16 at 10:13
Doing this in Python is an excellent question and doesn't deserve a downvote. (Also, handy command-line utilities like `jq` exist, so any Python solution will be somewhat more flabby than `jq`). — smci, Jul 31 '17 at 18:37

score 18 · Accepted Answer · edited Dec 16 '21 at 08:41

18

Based on the JSON below being returned from the curl command ...

'{"hostname":"test","domainname":"example.com"}'

You can then use python to extract the hostname using the python json module:

HOSTNAME=$(curl -s "$HOST" |
  python -c \
    'import json,sys;print json.load(sys.stdin)["hostname"]')

Note that I have split the line using a \ to make it more readable on stackoverflow. I've also simplified the command based on chepner's comment.

Original source: Parsing JSON with Unix tools

See also: https://wiki.python.org/moin/Powerful%20Python%20One-Liners

edited Dec 16 '21 at 08:41

tripleee

158,107
27
234
292

answered Jul 20 '16 at 17:36

Chris Snow

22,050
31
128
279

1

For a one-liner, I wouldn't bother with the intermediate variable `obj`: `python -c 'import json,sys;print json.load(sys.stdin)["hostname"]`. – chepner Jul 20 '16 at 17:50
2

For python3: `python3 -c 'import json,sys; print(json.load(sys.stdin)["hostname"])'`. Same thing but with parens for print – Xiao Dec 07 '20 at 02:20
This answer, while technically correct, is not conveniently formatted with executable code snippets. For instance most people are more interested in parsing the JSON, than in making a curl call to a service running on particular host. (I tried to edit it for clarity, but was rejected.) – MarkHu Feb 23 '21 at 19:41

score 11 · Answer 2 · answered Jan 10 '18 at 08:31

11

echo '{"hostname":"test","domainname":"example.com"}' | python -m json.tool

answered Jan 10 '18 at 08:31

hustljian

929
11
9

7

This only prettily prints the incoming json. There is no extraction of variables or array members. – Dan Locks Sep 27 '18 at 16:43
echo '{"hostname":"test","domainname":"example.com"}' | python -m json.tool | awk '/hostname/{ printf $2 }' | tr --delete '"' – Thilo Cestonaro Dec 07 '20 at 08:11

score 3 · Answer 3 · edited Dec 16 '21 at 08:40

Since Python is multiplatform, is important to note differences between Linux and Windows, especially because of how they treat double-quotes/single-quotes differently.

Second, some previous answers are a little bit obsolete: in python2, print without parentheses was permitted. Nevertheless, in python3, print must be between parentheses.

Linux (bash)

It doesn't matter how you put double/single quotes. Json can be parsed in both ways with "keys" or 'keys'

HOSTNAME=$(curl -s "$HOST" |
  python3 -c 'import json,sys;print(json.load(sys.stdin)["hostname"])')

It also works: (pay attention at single/double quote at key)

HOSTNAME=$(curl -s "$HOST" |
  python3 -c "import json,sys;print(json.load(sys.stdin)['hostname'])")

Windows (powershell)

Keys in json MUST be between single quotes. Only the following syntax is accepted.
The ConvertTo-Json function generates object and works with keys between single quotes.

$HOSTNAME=(Invoke-RestMethod $HOST | `
  ConvertTo-Json | `
  python3 -c "import json,sys; print(json.load(sys.stdin)['hostname'])")

This is a very nice answer to my question. Up voted. – Chris Snow Dec 03 '21 at 11:51 — Chris Snow, Dec 03 '21 at 11:51

how can I parse json with a single line python command?

3 Answers3

Linux (bash)

Windows (powershell)