16

What is the specific code, in order, being executed when I ask for something like

>>> 1 <= 3 >= 2
True

If both have equal precedence and it's just the order of their evaluation, why does the second inequality function as (3 >= 2) instead of (True >= 2)

Consider for example the difference between these

>>> (1 < 3) < 2
True

>>> 1 < 3 < 2
False

Is it just a pure syntactical short-cut hard-coded into Python to expand the second as the and of the two statements?

Could I change this behavior for a class, such that a <= b <= c gets expanded to something different? It's looking like the following is the case

a (logical operator) b (logical operator) c 
    --> (a logical operator b) and (b logical operator c)

but the real question is how this gets implemented in code.

I'm curious so that I can replicate this kind of __lt__ and __gt__ behavior in some of my own classes, but I am confused about how this is accomplished holding the middle argument constant.

Here's a specific example:

>>> import numpy as np

>>> tst = np.asarray([1,2,3,4,5,6])

>>> 3 <= tst
array([False, False,  True,  True,  True,  True], dtype=bool)

>>> 3 <= tst <= 5
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/home/ely/<ipython-input-135-ac909818f2b1> in <module>()
----> 1 3 <= tst <= 5

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

It would be nice to override this so that it "just works" with arrays too, like this:

>>> np.logical_and(3 <= tst, tst <= 5)
array([False, False,  True,  True,  True,  False], dtype=bool)

Added for clarification

In the comments it is indicated that I did a poor job of explaining the question. Here's some clarifying remarks:

1) I am not looking for a simple explanation of the fact that the interpreter pops an and in between the two chained inequalities. I already knew that and said so above.

2) For an analogy to what I want to do, consider the with statement (link). The following:

with MyClass(some_obj) as foo:
    do_stuff()

unpacks into

foo = MyClass(some_obj)
foo.__enter__()
try:
    do_stuff()
finally:
    foo.__exit__()

So by writing MyClass appropriately, I can do many special things inside of the with statement.

I am asking whether there is a similar code unpacking of the chained inequality by which I can intercept what it's doing and redirect it to use array-style logical operators instead just for the classes I care about.

I feel this is very clear from my question, especially the example, but hopefully this makes it more clear.

ely
  • 70,012
  • 31
  • 140
  • 215

3 Answers3

12

I'm not totally sure what you're looking for, but a quick disassembly shows that a < b < c is not compiled to the same bytecode as a < b and b < c

>>> import dis
>>>
>>> def f(a, b, c):
...     return a < b < c
...
>>> dis.dis(f)
  2           0 LOAD_FAST                0 (a)
              3 LOAD_FAST                1 (b)
              6 DUP_TOP
              7 ROT_THREE
              8 COMPARE_OP               0 (<)
             11 JUMP_IF_FALSE_OR_POP    21
             14 LOAD_FAST                2 (c)
             17 COMPARE_OP               0 (<)
             20 RETURN_VALUE
        >>   21 ROT_TWO
             22 POP_TOP
             23 RETURN_VALUE
>>>
>>> def f(a, b, c):
...     return a < b and b < c
...
>>> dis.dis(f)
  2           0 LOAD_FAST                0 (a)
              3 LOAD_FAST                1 (b)
              6 COMPARE_OP               0 (<)
              9 JUMP_IF_FALSE_OR_POP    21
             12 LOAD_FAST                1 (b)
             15 LOAD_FAST                2 (c)
             18 COMPARE_OP               0 (<)
        >>   21 RETURN_VALUE

Edit 1: Digging further, I think this is something weird or wrong with numpy. Consider this example code, I think it works as you would expect.

class Object(object):
    def __init__(self, values):
        self.values = values
    def __lt__(self, other):
        return [x < other for x in self.values]
    def __gt__(self, other):
        return [x > other for x in self.values]

x = Object([1, 2, 3])
print x < 5 # [True, True, True]
print x > 5 # [False, False, False]
print 0 < x < 5 # [True, True, True]

Edit 2: Actually this doesn't work "properly"...

print 1 < x # [False, True, True]
print x < 3 # [True, True, False]
print 1 < x < 3 # [True, True, False]

I think it's comparing boolean values to numbers in the second comparison of 1 < x < 3.

Edit 3: I don't like the idea of returning non-boolean values from the gt, lt, gte, lte special methods, but it's actually not restricted according to the Python documentation.

http://docs.python.org/reference/datamodel.html#object.lt

By convention, False and True are returned for a successful comparison. However, these methods can return any value...

FogleBird
  • 70,488
  • 25
  • 121
  • 130
  • This is not quite what I am looking for, but it is helpful and closer to the right track than the other answers so far. Since it doesn't yield the same byte code, the question is what Python functions are being inserted to make up the call in the first case, and how I can override them. – ely Oct 01 '12 at 14:26
  • Yes, you appear to be correct. This is a NumPy bug. For example, if I do the following: `list((-1 < vect)[:,0]) and list((vect < 5)[:,0])` then it works. So the `and` is choking on the fact that it's two arguments are `numpy.ndarray` instead of `list`. This is very odd; it means that `__gt__` and `__lt__` must have some extra `ndarray` cruft in them. – ely Oct 01 '12 at 14:42
  • It does appear to have issues with getting the right True/False value though, but at least it's not choking on the array types. Lots more digging to do. I wonder if this has anything to do with `__and__`. – ely Oct 01 '12 at 14:44
  • @EMS, what you want is simply impossible. You should use the binary operators instead which work fine (thats what `__and__` is). The `and` operator cannot do any element wise logic, it is simply not possible in Python, the result is what you find in **Edit 2**. Only the last comparison/object can be returned, and that will not do what you expect. – seberg Oct 04 '12 at 00:24
  • I think you're missing my point. I am asking how can I force the double sided inequality to expand into something that doesn't use the regular `and`. – ely Oct 04 '12 at 03:07
  • @EMS, well you obviously can't, if you look at the compiled code it does exactly the same (ignoring that its a bit optimized), both use the `JUMP_IF_FALSE_OP_POP` logic which is *exactly* the and operator. – seberg Oct 04 '12 at 08:41
  • I don't like the tone, but I could be misreading your comment. This was *not obvious* to me, which is why I asked the question. The very answer above illustrates the point you're trying to make, which is why I accepted it as the answer. I'm really not sure what your comment is meant to add. – ely Oct 04 '12 at 13:04
6

Both have the same precedence, but are evaluated from left-to-right according to the documentation. An expression of the form a <= b <= c gets expanded to a <= b and b <= c.

stochastic
  • 2,825
  • 4
  • 24
  • 40
Óscar López
  • 225,348
  • 35
  • 301
  • 374
  • Why doesn't it result in (1 <= 3) --> True --> (True >= 2) --> type error? – ely Sep 30 '12 at 02:46
  • 1
    The expression `a <= b <= c` gets expanded to `a <= b and b <= c` – Óscar López Sep 30 '12 at 02:50
  • 1
    In code, how does the expansion happen. Or is this just a one-off hard-coded Python idiom? – ely Sep 30 '12 at 02:52
  • See my expanded question above. – ely Sep 30 '12 at 03:00
  • Thanks again for your help on this. I do appreciate it, so please don't misread the following request. Would you be willing to delete your answer? It does not address any of the substantive points of my questions, and only points to the documentation for logical operators in isolation (which I had already read through before posting the question). I am concerned that because there is an answer now, the question won't get enough new attention. I will try adding a bounty, but if you could take down your answer too, that would be very helpful. Thank you. – ely Oct 01 '12 at 11:54
  • 1
    I really don't think removing this answer it going to get you better responses. Firstly, that's what voting is for, and secondly, continuing to improve your question will have more impact. – Useless Oct 01 '12 at 12:13
  • @EMS: what substantive points are you referring to? If you want to get something like a numpy array to "just work" with chained comparisons this way, then you'll need to override `__and__`. `array([0,5]) > 2` and `array([2,5]) < 10` both work and produce a bool-dtyped array, but `(array([0,5]) > 2) and (array([2,5]) < 10)` won't, because that expression is ambiguous and numpy decides to refuse the temptation to guess. You can't tell Python to use something other than `and` to unchain the comparison. – DSM Oct 01 '12 at 12:15
  • I'm saying that the syntax that expands the chained inequality into an `and` of the two separate inequalities should use NumPy `logical_and` function instead of Python's `and` operation. I want to override this so that the expansion works with NumPy's. @DSM, I think you're misunderstanding me. I know that natively there is the whole "truth value of an array is ambiguous" dealie. I'm asking how I can create inequalities that use the existing array logic tools to handle it appropriately. – ely Oct 01 '12 at 12:32
  • It should be pretty easy, if `a < 5` produces an array of Bools to also get `3 < a < 5` to similarly produce an array. If this is just hard-coded into the interpreter so that it must always use Python's native `and` when expanding the expression, then OK, but this answer doesn't even address that. And if it's hard-coded, that seems bad and inflexible and unpythonic, especially given how common this would be in NumPy. – ely Oct 01 '12 at 12:33
  • 1
    The generalization that `a <= b <= c` gets expanded to `a <= b and b <= c` is incorrect. – FogleBird Oct 01 '12 at 14:40
1

but the real question is how this gets implemented in code.

Do you mean how the interpreter transforms it, or what? You already said

a (logical operator) b (logical operator) c 
    --> (a logical operator b) and (b logical operator c)

so I'm not sure what you're asking here OK, I figured it out: no, you cannot override the expansion from a < b < c into (a < b) and (b < c) IIUC.


I'm curious so that I can replicate this kind of __lt__ and __gt__ behavior in some of my own classes, but I am confused about how this is accomplished holding the middle argument constant.

It depends which of a, b and c in the expression a < b < c are instances of your own class. Implementing your __lt__ and __gt__ and methods gets some of the way, but the documentation points out that:

There are no swapped-argument versions of these methods (to be used when the left argument does not support the operation but the right argument does)

So, if you want Int < MyClass < Int, you're out of luck. You need, at a minimum, MyClass < MyClass < Something (so an instance of your class is on the LHS of each comparison in the expanded expression).

Useless
  • 59,916
  • 5
  • 82
  • 126
  • 1
    So you are saying that the expansion of `a < b < c` into `a < b and b < c` is just *hard-coded* into the code for this part of the interpreter? There's no way to (aside from branching Python source ), for some classes, expand using `numpy.logical_and` instead, so that logical arrays are handled correctly? – ely Oct 01 '12 at 12:35
  • I don't understand why the answers are so dismissive of this. Can you give some advice for how I can improve my question? I feel it is a very valid point that we should be able to get things to combine with `and` however it makes the most sense for our classes. This is a very very common kind of operation in Python Pandas and in NumPy, so it's not like I'm asking for some very arcane array support. For example, immediately getting back an array of bools indicating where a date variable falls between a valid start and end date would be much better expressed as one chained inequality. – ely Oct 01 '12 at 13:28
  • Your question starts asking how a chained inequality works, and later says you _already know_ that, and say the real question is something else. So, by about 5 paragraphs in, I'm no longer sure what you're asking, and feel you wasted my time on something you already knew. Now, in the eighth comment to the first answer, and the second comment to the second answer, you've actually made it clear what you wanted to ask. If you feel the answers are dismissive of the question you _wanted_ to ask it's because they're based on the question you _actually_ asked. – Useless Oct 01 '12 at 13:43
  • ... if I have time, I'll try hacking your question into something more answerable. Otherwise, I'd recommend you ask a new question which is clearer. – Useless Oct 01 '12 at 13:44
  • Sorry if it was not clear. I wanted to ask how does it work one layer below just the interpreter's replacement of one syntax with another. I already knew that it popped the `and` in between. But think of the `with` statement. This is implemented with an `__enter__` then a `try` block, and then a `finally` with the `__exit__` call. You can do a lot of syntax manipulation by knowing how `with` works. I was hoping that chained inequalities similarly reduced down to a sequence of calls to some `__` type functions, with the handling of the chaining explicitly given in one of those functions. – ely Oct 01 '12 at 14:08
  • I also disagree that it wasn't clear. I gave an extremely specific example at the end of my question. – ely Oct 01 '12 at 14:09