Python: Merge two lists of dictionaries

Question

Given two lists of dictionaries:

>>> lst1 = [{id: 1, x: "one"},{id: 2, x: "two"}]
>>> lst2 = [{id: 2, x: "two"}, {id: 3, x: "three"}]
>>> merge_lists_of_dicts(lst1, lst2) #merge two lists of dictionary items by the "id" key
[{id: 1, x: "one"}, {id: 2, x: "two"}, {id: 3, x: "three"}]

Any way to implement merge_lists_of_dicts what merges two lists of dictionary based on the dictionary items' keys?

what if lst2[0] = {id: 2, x: "five"} or if lst2[0] = {id: 2, y: "y"} — alko, Oct 24 '13 at 09:14
What if the "id" is the same, but the value is not?, Why not use tuples if your dictionaries are only 1 item? I think you are using dictionaries wrong? — Inbar Rose, Oct 24 '13 at 09:17
@LostAvatar: how can you tell from the post? That's legal in both python2.x and in python3.x — inspectorG4dget, Oct 24 '13 at 09:18
@inspectorG4dget My possible answer would be different in python 3 — LostAvatar, Oct 24 '13 at 09:19
@LostAvatar: Makes sense. I'd just post both solutions, but that's me — inspectorG4dget, Oct 24 '13 at 09:21
@LostAvatar if you're thinking of ChainMap you'd also want to check whether views are acceptable or if copies are required... — Jon Clements, Oct 24 '13 at 09:25
So, this is not a duplicate at all. The two are very significantly and importantly different. — Slater Victoroff, Feb 27 '15 at 05:51
@InbarRose this is not a duplicate. It's merging a list of n dictionaries while the other is merging 2 dictionaries. — Thiago Figueiro, Sep 15 '16 at 07:20
@SlaterTyranus , InbarRose: totally right on that. I voted for reopen. — Wtower, May 24 '17 at 13:12

score 17 · Answer 1 · answered Oct 24 '13 at 09:32

17

Perhaps the simplest option

result = {x['id']:x for x in lst1 + lst2}.values()

This keeps only unique ids in the list, not preserving the order though.

If the lists are really big, a more realistic solution would be to sort them by id and merge iteratively.

answered Oct 24 '13 at 09:32

georg

204,715
48
286
369

1

This will return a `dict_values` in python 3. which is a tad different from the desired result whihc is a plain list of dictionaries – Kim Stacks Mar 01 '20 at 06:50
2

@KimStacks you can convert it to a list by doing: `list(result)` – shaahiin Mar 02 '20 at 12:16

score 16 · Answer 2 · answered Oct 24 '13 at 09:29

16

lst1 = [{"id": 1, "x": "one"}, {"id": 2, "x": "two"}]
lst2 = [{"id": 2, "x": "two"}, {"id": 3, "x": "three"}]

result = []
lst1.extend(lst2)
for myDict in lst1:
    if myDict not in result:
        result.append(myDict)
print result

Output

[{'x': 'one', 'id': 1}, {'x': 'two', 'id': 2}, {'x': 'three', 'id': 3}]

answered Oct 24 '13 at 09:29

thefourtheye

221,210
51
432
478

2

this one should be declared as an answer, how come 6 years later the author of the question didn't vote for this? – Suomynona Dec 05 '19 at 05:49

score 10 · Answer 3 · answered Oct 24 '13 at 09:28

10

One possible way to define it:

lst1 + [x for x in lst2 if x not in lst1]
Out[24]: [{'id': 1, 'x': 'one'}, {'id': 2, 'x': 'two'}, {'id': 3, 'x': 'three'}]

Note that this will keep both {'id': 2, 'x': 'three'} and {'id': 2, 'x': 'two'} as you did not define what should happen in that case.

Also note that the seemingly-equivalent and more appealing

set(lst1 + lst2)

will NOT work since dicts are not hashable.

answered Oct 24 '13 at 09:28

roippi

24,635
4
45
71

This is the most correct of all the answers as this will return a list as output in python3 – Kim Stacks Mar 01 '20 at 07:12

score 3 · Answer 4 · answered Nov 16 '20 at 16:40

BTW, you can use 'pandas' for such calculations:

>>> import pandas as pd
>>> 
>>> lst1 = [{"id": 1, "x": "one"}, {"id": 2, "x": "two"}]
>>> lst2 = [{"id": 2, "x": "two"}, {"id": 3, "x": "three"}]
>>> 
>>> lst1_df = pd.DataFrame(lst1)
>>> lst2_df = pd.DataFrame(lst2)
>>> lst_concat_df = pd.concat([lst1_df, lst2_df])
>>> lst_grouped_res_df = lst_concat_df.groupby(["id", "x"]).agg(sum)
>>> print(lst_grouped_res_df.reset_index().to_dict('records'))

Output:

[{'id': 1, 'x': 'one'}, {'id': 2, 'x': 'two'}, {'id': 3, 'x': 'three'}]

Python: Merge two lists of dictionaries

4 Answers4

Linked

Related