0

I have a ordered list with duplicates (objects that compare equal) and I want to remove the duplicates. As I want to keep the order, I cannot use set.

The duplicates in my list don't ocurr directly after each other and I want to keep the last ocurrence. In this related post, it seems like only the first ocurrence of a duplicate is kept.

As I simplified example, this is what I want:

list_with_duplicates = [1, 2, 1, 3, 2, 1]
list_without_duplicates = [3, 2, 1]

The only thing I can think of is a messy implementation using multiple iterations to first save the last ocurrence of each duplicate and then recreating the original list using the order of the last ocurring duplicates.

Community
  • 1
  • 1
CGFoX
  • 4,294
  • 5
  • 39
  • 69

5 Answers5

2

Use any answer you like in the linked duplicate (such as the top one), with one change: reverse your list before you iterate over it, and reverse the result when you're done.

def rem_rev(seq):
    seen = set()
    seen_add = seen.add
    return [x for x in seq[::-1] if not (x in seen or seen_add(x))][::-1]
    #                     ^^^^^^                                   ^^^^^^
    #                  or reversed(seq)
Community
  • 1
  • 1
TigerhawkT3
  • 46,954
  • 6
  • 53
  • 87
  • I don't really get the boolean expression to be honest. What is the `or seen_add(x)` for? I guess it somehow adds `x` to `seen` if it's not already there but I don't know how or what the return value for that would be. – CGFoX Dec 19 '16 at 08:41
  • 1
    @CGFoX - For each element in `seq[::-1]`, it only includes that element in the resulting list if it isn't in `seen`. If `x in seen` isn't the case, it checks the result of adding `x` to `seen`, which returns `None`. It's basically "include it if it's not there, or add it if it's not." – TigerhawkT3 Dec 19 '16 at 09:12
0

You can reverse a list using '::-1':

>>> result = []
>>> for item in l[::-1]:
...     if item not in result:
...             result.append(item)
... 
>>> result[::-1]
[3, 2, 1]
>>>
Maurice Meyer
  • 14,803
  • 3
  • 20
  • 42
0

The ordered dict solution looks quite neat:

>>> x = [1, 2, 1, 3, 2, 1]
>>>
>>> from collections import OrderedDict
>>> list(reversed(OrderedDict.fromkeys(reversed(x))))
[3, 2, 1]

or:

>>> list(OrderedDict.fromkeys(x[::-1]))[::-1]
[3, 2, 1]
Community
  • 1
  • 1
ekhumoro
  • 107,367
  • 18
  • 208
  • 308
0
>>> def keep_second(x, y):
>>>     return y
>>> [functools.reduce(keep_second, group) for _, group in itertools.groupby(sorted(items, key=key_func), key=key_func)
yihuang
  • 339
  • 2
  • 6
0

This could be a clean solution too:

x = [1, 2, 1, 3, 2, 1]
sorted(set(x), key=lambda i: x[::-1].index(i), reverse=True)
# [3, 2, 1]
Mayank Porwal
  • 31,737
  • 7
  • 30
  • 50