Parse JSON MSG in Python

Question

I am trying to parse a json MSG into a python dict.

For reference, the message is received from the Things Network with the python MQTT handler.

Here is the format I am receiving when I print the object

msg = MSG(variable_group=MSG(data0=0, data1=0, data2=0), variable2='name', variable3='num')

In its default state, I can access individual fields by msg.variable2 for example which provides 'name' but does not provide the variable name itself.

This is fine for a scenario in which I hardcode everything into my system, but I would like it to be a bit more adaptable and create new entries for variables as they come in.

Is there any way to parse this in such a way that I get both the data and the variable name?

Thanks!

EDIT:

From the input above, I would like to get a python dict containing the variable name and data.

dict = 
{
variable_group : MSG(data0=0, data1=0, data2=0),
variable2 : 'name',
variable3 : 'num'
}

Currently, I can access the data via a for loop and can print the variable names if I print the entire structure, but cannot access the variable names through a looping mechanism

EDIT 2:

After doing some digging on the wrapper found the following:

def _json_object_hook(d):
    return namedtuple("MSG", d.keys())(*d.values())


def json2obj(data):
    return json.loads(data, object_hook=_json_object_hook)

Where the input shown above is created by passing it as 'data' to json2obj.

I am still unsure how to get a dict out of this format, haven't used object_hooks before.

From the input above, the desired output would be a python dict (or other structure) formatted as {variable_group : MSG(data.....), variable2 : 'name', variable3 : 'num' — tvanfossen, May 23 '18 at 18:33
In short you want to access like `msg.name`? and name in the not the key in the dictionary but value right? — mad_, May 23 '18 at 18:38
No, updated above. I want to be able to access the variable names which I cannot do as they are formatted in the input. Accessing data is currently not a problem — tvanfossen, May 23 '18 at 18:41
Just tried msg.__dict__, not an attribute of the "MSG" object. — tvanfossen, May 23 '18 at 18:43
I am using the ttn library for LoraWan comms. Python SDK here https://github.com/TheThingsNetwork/python-app-sdk — tvanfossen, May 23 '18 at 18:55
The message is created in the ttnmqtt.py under the MQTT client — tvanfossen, May 23 '18 at 18:58

theodox · Accepted Answer · 2018-05-23T19:17:57.250

0

From discussion in the comments below, it appears that the MSG object is a namedtuple created on the fly out of the json object.

In a case like that you can get the fields by looking at the _fields of the object. You can dict-ify a namedtuple like this

def nt_to_dict(nt):
     return {field, getattr(nt, field) for field in nt._fields}

or you could just inspect the object by trolling _fields in code and using getattr as needed

edited May 23 '18 at 19:17

answered May 23 '18 at 18:44

theodox

11,783
3
21
36

It is not a regular json blob as far as I can tell. I have tried both loads and load without a successful conversion. It is referred to in the errors as a 'MSG' object – tvanfossen May 23 '18 at 18:47
you could brute force reflection by looking at the `__dict__` of the MSG object. – theodox May 23 '18 at 18:52
Already tried that from a response in the comments above. Spits back that __dict__ is not an attribute of the MSG object – tvanfossen May 23 '18 at 18:56
ouch, an old style class? Or some kind of C wrapper object? – theodox May 23 '18 at 18:58
Not 100% sure. Its definitely a json format. Best I can find is this line obj = json2obj(j_msg) from inside the sdk – tvanfossen May 23 '18 at 19:01
After doing a bit more digging on that wrapper, found that the json2obj is is a util function I missed. It does: json.loads(data, object_hook=_json_object_hook) where _json_object_hook returns namedtuple("MSG", d.keys())(*d.values())... Any idea how to perform that in reverse? – tvanfossen May 23 '18 at 19:11
Had to fiddle with it a bit more to account for the nested namedtuples, but it works. Thanks! – tvanfossen May 23 '18 at 19:36
Suggestion: [`namedtuple._asdict()`](https://docs.python.org/2/library/collections.html#collections.somenamedtuple._asdict), as in `def nt_to_dict(nt): return dict(nt._asdict())` – Robᵩ May 23 '18 at 20:56

Parse JSON MSG in Python

1 Answers1

Linked