32

When programming in python, I now avoid map, lambda and filter by using list comprehensions because it is easier to read and faster in execution. But can reduce be replaced as well?

E.g. an object has an operator union() that works on another object, a1.union(a2), and gives a 3rd object of same type.

I have a list of objects:

L = [a1, a2, a3, ...]

How to have the union() of all these objects with list comprehensions, the equivalent of:

result = reduce(lambda a, b :a.union(b), L[1:], L[0])
Martijn Pieters
  • 963,270
  • 265
  • 3,804
  • 3,187
Eric H.
  • 1,962
  • 3
  • 20
  • 33
  • 1
    In some cases: no. But depends. Please provide a specific query that you have in mind – sshashank124 Jun 25 '14 at 13:44
  • 1
    @sshashank124 - any examples? – mhawke Jun 25 '14 at 13:53
  • 1
    Set unions are a bad example, because you can simply do `result = set().union(*L)`, which has the bonus of working even if L is an empty list. At any rate, `lambda a, b :a.union(b)` can be written more concisely as `set.union`, since in python `obj.method(args)` is the same as `cls.method(obj, args)` – Eric Mar 11 '16 at 07:07
  • Guido says to use a for loop instead of reduce. He's not a fan of FP constructs. – Didier A. Nov 04 '16 at 20:32

5 Answers5

20

It is no secret that reduce is not among the favored functions of the Pythonistas.

Generically, reduce is a left fold on a list

It is conceptually easy to write a fold in Python that will fold left or right on a iterable:

def fold(func, iterable, initial=None, reverse=False):
    x=initial
    if reverse:
        iterable=reversed(iterable)
    for e in iterable:
        x=func(x,e) if x is not None else e
    return x

Without some atrocious hack, this cannot be replicated in a comprehension because there is not accumulator type function in a comprehension.

Just use reduce -- or write one that makes more sense to you.

dawg
  • 90,796
  • 20
  • 120
  • 197
8

Since a list comprehension definitionally generates another list, you can't use it to generate a single value. The aren't for that. (Well... there is this nasty trick that uses a leaked implementation detail in old versions of python that can do it. I'm not even going to copy the example code here. Don't do this.)

If you're worried about the stylistic aspects of reduce() and its ilk, don't be. Name your reductions and you'll be fine. So while:

all_union = reduce(lambda a, b: a.union(b), L[1:], L[0])

isn't great, this:

from functools import reduce

def full_union(input):
    """ Compute the union of a list of sets """
    return reduce(set.union, input[1:], input[0])

result = full_union(L)

is pretty clear.

If you're worried about speed, check out the toolz and cytoolz packages, which are 'fast' and 'insanely fast,' respectively. On large datasets, they'll often let you avoid processing your data more than once or loading the whole set in memory at once, in contrast to list comprehensions.

Nate
  • 4,313
  • 2
  • 30
  • 41
  • 2
    To make the `reduce()` expression itself readable, make the first argument not a lambda. For example: `reduce(set.union, )` Sometimes this will require you to define (and therefore name) the operator somewhere outside the call to `reduce`. – Jordan Oct 11 '17 at 00:12
5

A common use of reduce is to flatten a list of lists. You can use a list comprehension instead.

L = [[1, 2, 3], [2, 3, 4], [3, 4, 5]]

with reduce

from functools import reduce  # python 3
flattened = reduce(lambda x, y: x + y, L)

print(flattened)

[1, 2, 3, 2, 3, 4, 3, 4, 5]

with list comp

flattened = [item for sublist in L for item in sublist]

print(flattened)

[1, 2, 3, 2, 3, 4, 3, 4, 5]

If your problem can be solved by operating on the flattened list, this is an effective replacement. Contrast these one-liners for the given example:

all_union = reduce(lambda a, b: set(a).union(set(b)), L)

{1, 2, 3, 4, 5}

all_union = set([item for sublist in L for item in sublist])

{1, 2, 3, 4, 5}
Nicholas Morley
  • 3,262
  • 2
  • 15
  • 14
  • 3
    Use `sum(L, [])`. That said, the sum/list comprehension creates a "full-length" list, where `reduce(operator.or_, map(set, L), set())` would not. – Andy Hayden Nov 09 '17 at 06:02
4

Not really. List comprehensions are more similar to map and filter.

hd1
  • 32,598
  • 5
  • 75
  • 87
bigblind
  • 12,015
  • 13
  • 68
  • 115
0

Not the most readable reduce for unions, but assignment expressions can be used to solve it in a single expression:

In [51]: L = [{1}, {2}, {3}, {4}, {5}]

In [52]: reduce(lambda a, b: a.union(b), L[1:], L[0])
Out[52]: {1, 2, 3, 4, 5}

In [53]: (s := set(), [s := s.union(x) for x in L][-1])[1]
Out[53]: {1, 2, 3, 4, 5}
dawid
  • 591
  • 3
  • 10