7

Adding or multiplying a large list of numbers in Python can elegantly be done by folding the list with the addition or multiplication operator:

import functools, operator
lst = range(1,100)
sum  = functools.reduce(operator.add, lst)
prod = functools.reduce(operator.mul, lst)

This needs the function equivalents of the operators + and * which are provided by the operator module as operator.add and operator.mul, respectively.

If I want to use the same idiom with the operator or:

ingredients       = ['onion', 'celery', 'cyanide', 'chicken stock']
soup_is_poisonous = functools.reduce(operator.or, map(is_poisonous, ingredients))

... then I discover that operator doesn't have a function equivalent of the logical and and or operators (though it has one for logical not)

Of course, I can trivially write one that works:

def operator_or(x,y):
  return x or y

But I wonder: why are there no operator.or and operator.and in operator? Bitwise and and or are there, but not the logical ones.

Of course this is just a minor annoyance, and the answer may well be the same as with the missing identity function: that it is easy to write one. But this holds for * and + as well, so why the difference?

Hans Lub
  • 5,513
  • 1
  • 23
  • 43
  • 1
    `functools.reduce()` doesn't provide a way to signal that you want to short circuit the operation, so it doesn't seem like there would be a difference between logical vs bitwise `and`/`or`. – rchome Feb 04 '22 at 11:03
  • 2
    So what would a hypothetical logical-and function look like? Remember that it must only evaluate its right-hand side if necessary. (Before someone points it out: Yes, I suppose it could take the second argument in the form of a function to call. I'm not sure how much contortions that would require from the compiler, but apparently someone thought it not worth the trouble...) – Ture Pålsson Feb 04 '22 at 11:04
  • @rchome - Exactly what I was thinking, all operands would be evaluated, which is not the same as `and` and `or` in most languages. (I don't do Python and struggle with how its documentation is organized, so couldn't readily find out if it's also true in Python, but it seemed likely. *Edit:* [found it](https://docs.python.org/3/reference/expressions.html#boolean-operations)) – T.J. Crowder Feb 04 '22 at 11:04
  • Yes, that is probably it (see my comment below timgeb's answer). Coming from Haskell, I was slow to realize that Python is a [strict programming language](https://en.wikipedia.org/wiki/Strict_programming_language) – Hans Lub Feb 04 '22 at 11:53
  • @rchome There are differences. `1 and 2` is true but `1 & 2` is false. And `[1] | [2]` errors. – Kelly Bundy Feb 05 '22 at 00:42
  • @Kelly `1 and 2` is in fact `2`…! – deceze Feb 13 '22 at 12:36
  • @deceze Can you clarify what you mean, why you're saying that? – Kelly Bundy Feb 13 '22 at 15:34
  • @Kelly `1 and 2` returns the value `2`, not `True`, as stated or implied by you. – deceze Feb 13 '22 at 19:42
  • @deceze I did **not** state or imply that it's `True`. I said it's true. Which it is. – Kelly Bundy Feb 13 '22 at 20:20
  • @Kelly *Truthy?* – deceze Feb 13 '22 at 21:19
  • @deceze What about that? That's an abominable word, and not official terminology. See for example [this recent issue](https://bugs.python.org/issue45346), where a few "truthy" that had crept in were immediately eradicated, without debate. Official terminology is true/false, as mentioned in that issue, also see the five links there. – Kelly Bundy Feb 13 '22 at 21:48

2 Answers2

3

To wrap up all your helpful answers and comments, in order of somewhat decreasing (to me) convincingness:

  1. the addition of operator.or would break an important promise made by the module

    For all operators <op> that have function equivalents operator.op in the operator module, it is the case that a <op> b is equivalent to (i.e. can always, without changing program behaviour, replace or be replaced by) operator.op(a, b). This equivalence is actually mentioned in the module docstring. This is impossible to do for the operators and and or as their evaluation is short-circuiting while Python function calls are always evaluated after all of their arguments are.

  2. On the values True and False, | and &, hence also the existing (bitwise) operator.and_ and operator.or_ already return the same results (if they return at all, that is) as or and and.

    If is_poisonous() returns either True of False (not an unreasonable requirement), I could use

    soup_is_poisonous = reduce(operator.or_, map(is_poisonous, ingredients), False)

    in the example from the original question. However, many Python programs conveniently use any "truthy" value as True in idioms like

    your_model_T_color = "black" or any_color_you_like

    using | or operator.or_ instead of or here will result in a TypeError or, even worse, some unexpected value (if the operands are ints)

  3. The functions any and all can be used instead of functools.reduce(operator.or, ....)

    I'm not convinced by this argument: operator functions are used in many more contexts than as a first argument to reduce. Moreover, any always returns either True or False, not the first truthy value:

    any([0,0,0,5,6,7]) # returns True
reduce(lambda x, y: x or y, [0,0,0,5,6,7]) # returns 5

    so any and reduce(operator.or would not really be equivalent

  4. any([x,y]) does the same (and more, as it accepts iterables) as operator.or(x,y) would.

    That is not quite true (see above), any([0,5]) returns True while operator.or(0,5) would return 5. Moreover, the number of arguments matters greatly if we use a function as an argument to another function like reduce()

Hans Lub
  • 5,513
  • 1
  • 23
  • 43
2

all is short-circuiting logical-and.
any is short-circuiting logical-or.

No need to put versions that take exactly two arguments (instead of an iterable) into the operator module, I guess.

timgeb
  • 76,762
  • 20
  • 123
  • 145
  • That there are excellent alternatives for the idiom `reduce(operator.or, ...)` doesn't make `operator.or` superfluous, as there may be other uses for such a function. But that such a function will necessarily behave differently from the operator (as Python _always_ evaluates function arguments before calling a function) is probably a good reason to leave it out and thus avoid confusion. Forcing me to write `operator_or` myself makes the difference explicit and is probably a wise design choice. – Hans Lub Feb 04 '22 at 11:44
  • They don't do the same thing, though, always returning `True` or `False` instead of one of the operands. – Kelly Bundy Feb 05 '22 at 00:38
  • `all` and `any` are not (and _cannot_) be short-circuiting. `any([x == 0, 1/x > 1])` will divide by zero if `x == 0`, while `x == 0 or 1/x > 1` will not. – Hans Lub Feb 13 '22 at 10:34
  • @HansLub the functions any and all are understood to be [short-circuiting](https://stackoverflow.com/questions/14730046/is-the-shortcircuit-behaviour-of-pythons-any-all-explicit). As you, I and everybody else knows function argument evaluation is never lazy in Python. – timgeb Feb 13 '22 at 17:18
  • 1
    I think we are arguing about words here. As I understand it, when you call `any()` short-circuiting, you mean that for any iterator `it`, `and(it)` will call `it.__next__()` often enough, but no more, to determine a return value. I call a function or operator [short-circuiting](https://en.wikipedia.org/wiki/Short-circuit_evaluation) if it evaluates just the arguments it needs to determine a return value. Python's `and` and `or` are short-circuiting in one sense of the word, while `any()` and `all()` are short-cicuiting in another. – Hans Lub Feb 13 '22 at 17:36
  • @HansLub sure. There's only one way functions can short circuit in Python, so there's no ambiguity in my statement. It's different in other languages. – timgeb Feb 13 '22 at 17:42