I want to remove all duplicates list from a list of list.
So I have a list of lists like this.
a = [[1,2],[1,2],[3,4,5],[3,4,5],[3,4,5]]
I want to have:
b = [[1,2],[3,4,5]]
I don't know how to do it.
Asked
Active
Viewed 4.1k times
37
martineau
- 112,593
- 23
- 157
- 280
user1507156
- 519
- 1
- 7
- 12
2 Answers
78
You could use a set:
b_set = set(map(tuple,a)) #need to convert the inner lists to tuples so they are hashable
b = map(list,b_set) #Now convert tuples back into lists (maybe unnecessary?)
Or, if you prefer list comprehensions/generators:
b_set = set(tuple(x) for x in a)
b = [ list(x) for x in b_set ]
Finally, if order is important, you can always sort b:
b.sort(key = lambda x: a.index(x) )
mgilson
- 283,004
- 58
- 591
- 667
-
1If you want something to be unique, a set is the way to go. :) – mgilson Aug 30 '12 at 13:54
-
`lambda x: a.index(x)` could be just `a.index` – Jean-François Fabre Oct 26 '18 at 19:01
-
1`b = map(list,b_set)` will return a map of lists, if you want it back to the original list of lists you'll want to instead do `b = list(map(list,b_set))` – defermat Apr 19 '19 at 18:01
15
See mgilson's answer if the order of the lists is not important. If you want to retain the order, do something like:
b = list()
for sublist in a:
if sublist not in b:
b.append(sublist)
This will keep the order in the original list. However, it is slower and more verbose than using sets.
HerrKaputt
- 2,584
- 16
- 17
-
Thank you too, your method is interesting, but I don't need to keep the order. – user1507156 Aug 30 '12 at 14:12
-
Just note that this method will treat [1,2] and [2,1] as different, though it works for this example. – Zoe L Aug 01 '18 at 23:28
-