Combining list elements in python?

Question

Possible Duplicate:
How do you remove duplicates from a list in Python whilst preserving order?

this question might be quite easy. For example, I ve got a list like that :

a = [1,1,1,1,2,3,4,5,5,5,5,6,7,7,8,9,14,14]

I dont want to hold the same elements in my list. So, i d like to transform it to:

a = [1,2,3,4,5,6,7,8,9,14]

How can i manage this ? Thank you in advance.

And then the followup question is *do you even want a list, or just an iterable?* Because in the latter case, why convert it back to a list? `set(a)` and you're home. — kojiro, Jan 23 '13 at 22:06
yes, the order does matter but i am going to work with ordered lists already. so, under this condition, does "list(set(a))" always give me ordered output? — user1907576, Jan 23 '13 at 22:10
@user1907576: no, `set`s do not preserve input order. See the linked duplicate for more ways than you care for to remove duplicates from your list while preserving ordering, at various speeds. — Martijn Pieters, Jan 23 '13 at 22:11
@user1907576: Besides not trying anything initially, you apparently don't realize that you can also answer your follow-on own questions by just trying things out and seeing what happens. — martineau, Jan 23 '13 at 22:17

score 4 · Accepted Answer · answered Jan 23 '13 at 22:06

4

Transform it to a set:

a = list(set(a))

answered Jan 23 '13 at 22:06

Emanuele Paolini

9,365
3
35
60

This does not preserve order – Matt Jan 23 '13 at 22:23

Ashwini Chaudhary · Answer 2 · 2013-01-23T22:17:48.093

You can use unique_everseen recipe from itertools, it maintains the order.

If order doesn't matter then use set().

In [294]: a = [1,1,1,1,2,3,4,5,5,5,5,6,7,7,8,9,14,14]

In [295]: list(unique_everseen(a))
Out[295]: [1, 2, 3, 4, 5, 6, 7, 8, 9, 14]

unique_everseen:

def unique_everseen(iterable, key=None):
    "List unique elements, preserving order. Remember all elements ever seen."
    # unique_everseen('AAAABBBCCDAABBB') --> A B C D
    # unique_everseen('ABBCcAD', str.lower) --> A B C D
    seen = set()
    seen_add = seen.add
    if key is None:
        for element in ifilterfalse(seen.__contains__, iterable):
            seen_add(element)
            yield element
    else:
        for element in iterable:
            k = key(element)
            if k not in seen:
                seen_add(k)
                yield element

score 2 · Answer 3 · answered Jan 23 '13 at 22:06

2

Converting the list to a set will remove the duplicates, and then convert it back to a list.

a = list(set(a))

answered Jan 23 '13 at 22:06

Diego Allen

4,543
4
30
33

score 1 · Answer 4 · answered Jan 23 '13 at 22:08

1

import itertools
a = [k for k, g in itertools.groupby(a)]

This assumes that the duplicate items in the original list are already grouped, as in your example. The benefit of this method over using set() is that the original order will be maintained.

answered Jan 23 '13 at 22:08

Andrew Clark

192,132
30
260
294

score 0 · Answer 5 · answered Jan 23 '13 at 22:09

0

Or, If you need to preserve sorted order:

a = [1,1,1,1,2,3,4,5,5,5,5,6,7,7,8,9,14,14]
sorted(set(a))

Results in:

[1, 2, 3, 4, 5, 6, 7, 8, 9, 14]

answered Jan 23 '13 at 22:09

jimhark

4,783
1
25
26

That only works if the input was sorted in the first place. – Martijn Pieters Jan 23 '13 at 22:10
This only preserves the order if the list was originally sorted. – Diego Allen Jan 23 '13 at 22:10
Changed description. His example list was sorted, but my answer is more accurate with the edit. Thanks. – jimhark Jan 23 '13 at 22:14

Combining list elements in python?

5 Answers5