Python dictionary manipulation into list of dictionary

Question

I m fairly new to python and having a hard time solving this problem, I have a dictionary in format:

{'tag1_batch1_st': OrderedDict([(0.0, '59.83163071'), (1.0, '18.94903946'), (2.0, '48.38692856'), (3.0, '0.05321884'), (4.0, '4.20608902'), (5.0, '01.66112137'), (6.0, '0.59775162'), (7.0, '37.17335892'), (8.0, '0.78234863'), (9.0, '1.67506027')]), 'tag_batch_st ': OrderedDict([(0.0, '58.83163071'), (1.0, '48.94903946'), (2.0, '38.38692856'), (3.0, '45.05321884'), (4.0, '40.20608902'), (5.0, '38.66112137'), (6.0, '37.59775162'), (7.0, '37.17335892'), (8.0, '36.78234863'), (9.0, '36.67506027')])}

I want to convert it into a list like this:

[{'tag1_batch1_st': '59.83163071', 'num': 0.0, 'tag_batch_st': '58.83163071'}, ...... ]

I know, I can use for loop and iterate over the dictionary and create the list, but any approach which would require less iterations and methods like groupby() would be helpful.

Thanks

Your example data could be laid out better to make what you want easier to see — SpoonMeiser, Nov 27 '17 at 10:45

Moses Koledoye · Accepted Answer · 2017-11-27T12:45:26.867

3

There's no approach that'll give a complexity less than O(n) - n is the number of key, value pairs in the OrderedDict. You have iterate on every pair, so no such thing as less iterations.

You can iterate on the dict values' key-value pairs (i.e. key-value pairs of OrderDict) simultaneously by zipping them and then build new dicts using the new pair and adding a new key entry:

lst = []
for (num, st1), (_, st2) in zip(*map(dict.items, dct.values())):
   d = dict(zip(dct.keys(), (st1, st2)))
   d['num'] = num
   lst.append(d)

[{'tag1_batch1_st': '59.83163071', 'tag_batch_st ': '58.83163071', 'num': 0.0}, 
 {'tag1_batch1_st': '18.94903946', 'tag_batch_st ': '48.94903946', 'num': 1.0}, 
 ...
]

In case you have more keys in the dictionary, you can use map(zip, ...) to collect all the values from the ordered dicts in a tuple, and the zip the tuple with the parent dictionary keys:

for (num, _), sts in map(zip, *map(dict.items, dct.values())):
   d = dict(zip(dct.keys(), sts))
   d['num'] = num
   lst.append(d)

edited Nov 27 '17 at 12:45

answered Nov 27 '17 at 10:45

Moses Koledoye

74,909
8
119
129

Does not give the desired output, both the keys 'k' and 'tag_batch_st', have the same value. But, if you look at the output, the output is different. – Shivin Agarwal Nov 27 '17 at 10:55
1

@ShivinAgarwal Didn't notice. So why keep us guessing about how the value for `'tag_batch_st'` was created? – Moses Koledoye Nov 27 '17 at 10:57
its a database dump. – Shivin Agarwal Nov 27 '17 at 10:59
It throws error: ' TypeError: items() takes no arguments (1 given) ' – Shivin Agarwal Nov 27 '17 at 11:20
@ShivinAgarwal You've assigned the builtin `dict` using `dict = ..` somewhere. You shouldn't do that. – Moses Koledoye Nov 27 '17 at 11:21
Now its working, after your edit. But can you please explain what you did? or provide a link for the same. Thank you. – Shivin Agarwal Nov 27 '17 at 11:28
but, what if it contained more keys like 'tag2_batch2_st' and so on, which would be dynamic in nature, then? – Shivin Agarwal Nov 27 '17 at 11:58
@ShivinAgarwal I've updated the answer with code handling such cases. – Moses Koledoye Nov 27 '17 at 12:45
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/159901/discussion-between-shivin-agarwal-and-moses-koledoye). – Shivin Agarwal Nov 27 '17 at 13:05

score 0 · Answer 2 · answered Nov 29 '17 at 17:17

You can use default dict and lambda :

Two line solution :

from collections import OrderedDict,defaultdict
a={'tag1_batch1_st': OrderedDict([(0.0, '59.83163071'), (1.0, '18.94903946'), (2.0, '48.38692856'), (3.0, '0.05321884'), (4.0, '4.20608902'), (5.0, '01.66112137'), (6.0, '0.59775162'), (7.0, '37.17335892'), (8.0, '0.78234863'), (9.0, '1.67506027')]), 'tag_batch_st ': OrderedDict([(0.0, '58.83163071'), (1.0, '48.94903946'), (2.0, '38.38692856'), (3.0, '45.05321884'), (4.0, '40.20608902'), (5.0, '38.66112137'), (6.0, '37.59775162'), (7.0, '37.17335892'), (8.0, '36.78234863'), (9.0, '36.67506027')])}

default_dict = defaultdict(list)
list(map(lambda x:[default_dict[key].append(value) for key,value in x.items()],a.values()))


print([{'tag1_batch1_st':value[0],'tag_batch1_st':value[1],'num':key} for key,value in default_dict.items()])

output:

[{'num': 0.0, 'tag_batch1_st': '59.83163071', 'tag1_batch1_st': '58.83163071'}, {'num': 1.0, 'tag_batch1_st': '18.94903946', 'tag1_batch1_st': '48.94903946'}, {'num': 2.0, 'tag_batch1_st': '48.38692856', 'tag1_batch1_st': '38.38692856'}, {'num': 3.0, 'tag_batch1_st': '0.05321884', 'tag1_batch1_st': '45.05321884'}, {'num': 4.0, 'tag_batch1_st': '4.20608902', 'tag1_batch1_st': '40.20608902'}, {'num': 5.0, 'tag_batch1_st': '01.66112137', 'tag1_batch1_st': '38.66112137'}, {'num': 6.0, 'tag_batch1_st': '0.59775162', 'tag1_batch1_st': '37.59775162'}, {'num': 7.0, 'tag_batch1_st': '37.17335892', 'tag1_batch1_st': '37.17335892'}, {'num': 8.0, 'tag_batch1_st': '0.78234863', 'tag1_batch1_st': '36.78234863'}, {'num': 9.0, 'tag_batch1_st': '1.67506027', 'tag1_batch1_st': '36.67506027'}]

Python dictionary manipulation into list of dictionary

2 Answers2