Removing duplicate strings from a list in python

Question

If I have a string list ,

a = ["asd","def","ase","dfg","asd","def","dfg"]

how can I remove the duplicates from the list?

score 87 · Accepted Answer · answered Nov 20 '11 at 08:32

87

Convert to a set:

a = set(a)

Or optionally back to a list:

a = list(set(a))

Note that this doesn't preserve order. If you want to preserve order:

seen = set()
result = []
for item in a:
    if item not in seen:
        seen.add(item)
        result.append(item)

See it working online: ideone

answered Nov 20 '11 at 08:32

Mark Byers

767,688
176
1,542
1,434

thank you :) that worked .... – Shamika Feb 18 '12 at 23:30

score 12 · Answer 2 · answered Nov 20 '11 at 08:32

12

Use the set type to remove duplicates

a = list(set(a))

answered Nov 20 '11 at 08:32

Krumelur

29,183
6
73
115

murasing · Answer 3 · 2014-02-07T09:59:49.943

6

def getUniqueItems(iterable):
result = []
for item in iterable:
    if item not in result:
        result.append(item)
return result

print (''.join(getUniqueItems(list('apple'))))

P.S. Same thing like one of the answers here but a little change, set is not really required !

edited Feb 07 '14 at 09:59

answered Feb 07 '14 at 09:49

murasing

412
6
15

does this code snippet preserve the order? – tumultous_rooster Jun 09 '14 at 06:16
It does preserve order, but has quadratic complexity, and that is bad for any list remotely big. – ReneSac Mar 29 '15 at 03:35
this awesome! thanks – raditya gumay Sep 24 '16 at 14:57

score 5 · Answer 4 · answered Nov 20 '11 at 08:33

5

You could pop 'em into a set and then back into a list:

a = [ ... ]
s = set(a)
a2 = list(s)

answered Nov 20 '11 at 08:33

rossipedia

52,494
10
88
90

Removing duplicate strings from a list in python

4 Answers4

Linked