Faster loops and booleans in Python, without numba

Question

I seek something that speeds up plain for loops, arithmetic, and booleans - no np. calls except maybe for passing np.array() into the function. Numba is rather heavy on download size and compile time and I try to avoid it as a dependency for some of my libraries.

Is there such a lightweight alternative with restricted functionality (besides compiled C etc, must stay in Python)? Example function:

A, B, i = 0, 0, 0
while i < 1000:
    if A > 5:
        A += x[i]
        B = min(B, A / 2)
    else:
        A -= x[i]
    i += 1

Voting to close as questions asking for recommendations for tools or libraries are explicitly off-topic for stack overflow. That said, what about [threading](https://docs.python.org/3/library/threading.html) or [multiprocessing](https://docs.python.org/3/library/multiprocessing.html)? Both are built-in so you avoid downloads and neither is compiled — G. Anderson, Jul 01 '22 at 19:15
@G.Anderson It's a variation of [this question](https://stackoverflow.com/q/64540868/10133797). Multiprocessing is pain and can't assist sequential computations; it's more about overcoming slowdowns due to typing. — OverLordGoldDragon, Jul 01 '22 at 19:19
If Numba is a problem, consider using Cython. Cython is not needed for users, only for developers. Users get the natively compiled package that run fast. If you do not want this either then you are stuck to the slow CPython interpreter and *Python is just not the right tool*. Python is designed for prototyping and scripting not heavy low-level math computations. Note that the loop cannot be efficiently vectorized so Numpy is quite useless here. Besides, why do you tag the question `numba` if you explicitly do not want it? — Jérôme Richard, Jul 01 '22 at 19:52
@JérômeRichard I thought "Cython" meant write in C and ship in Python... Perfect, thank you. — OverLordGoldDragon, Jul 02 '22 at 18:03

hpaulj · Answer 1 · 2022-07-01T20:06:11.440

In [55]: %%timeit
    ...: x = list(range(1000))
    ...: A, B, i = 0, 0, 0
    ...: while i < len(x):
    ...:     if A > 5:
    ...:         A += x[i]
    ...:         B = min(B, A / 2)
    ...:     else:
    ...:         A -= x[i]
    ...:     i += 1
    ...:     
315 µs ± 6.75 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

Replacing the while with a for gives a noticeable increase:

In [57]: %%timeit
    ...: x = list(range(1000))
    ...: A, B, i = 0, 0, 0
    ...: for i in range(len(x)):
    ...:     if A > 5:
    ...:         A += x[i]
    ...:         B = min(B, A / 2)
    ...:     else:
    ...:         A -= x[i]
143 µs ± 318 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

And a bit better with direct iteration on x:

In [59]: %%timeit
    ...: x = list(range(1000))
    ...: A, B = 0, 0
    ...: for v in x:
    ...:     if A > 5:
    ...:         A += v
    ...:         B = min(B, A / 2)
    ...:     else:
    ...:         A -= v
    ...:         
121 µs ± 324 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

In the right cases, list comprehension can further streamline the loop - but not here.

numpy vectorize makes use of compiled numpy methods, and numpy specific storage. np.vectorize clearly states it is not a performance tool. For large arrays it can improve on the speed of a list comprehension by a bit. But it's no use here, for the same reason a list comprehension doesn't work.

There aren't any other ways of 'streamlining' python loops.

This was a dummy example, [actual func](https://replit.com/@OverLordGoldDra/UnacceptableWindingExtension#main.py). Sometimes [C](https://pastebin.com/x8PS9VvA) is the only way. — OverLordGoldDragon, Jul 02 '22 at 18:06
The iteration method becomes less significant when the body is more complex and time consuming. Without compiling there's no way around the cost of performing the task N times. — hpaulj, Jul 02 '22 at 18:56
Right, I was under impression that "compiled C" necessarily means "write in C". — OverLordGoldDragon, Jul 02 '22 at 19:08
cython and numba meet you more than half way there half way there. — hpaulj, Jul 02 '22 at 19:54

Faster loops and booleans in Python, without numba

1 Answers1