Optimizing python code monte carlo simulation

Question

I'm trying to perform Monte Carlo simulation on Ising model. My code runs pretty slow due to many monte carlo steps I have to perform. I'm trying to optimize my code to make it faster. I already did optimze metropolis() function but I don't know how to optimize my simulation() function. I tried list comprehension, but can't figure out a smart way to use it. Any tips and help would be appreciated.

Below is my code.


def simulation(MCS, T, lattice, k0):

    values = {"m" : [], "C" : []}
    
    for n in range(MCS):
        
        lattice = metropolis(lattice, T_reduced)

        if n >= k0 and n % 1000 == 0:
            # keep track on values
            values["m"].append(some_function(lattice))
            values["C"].append(some_function(lattice))

    return values, lattice

Determine what is slow first. Use `cProfile` or something similar. Anything before that is premature optimization. — anon01, May 24 '21 at 09:02
Have you tried using @numba.jit on any of your functions? https://numba.pydata.org/numba-doc/latest/user/5minguide.html — Tom McLean, May 24 '21 at 11:56

score 1 · Accepted Answer · answered May 24 '21 at 08:56

Instead of using dict to store values of m and C define both as distinct variables and make the dict only when returning:

m = []
C = []
...
return {"m": m, "C": C}, lattice

You could also use a deque instead of a list, which should be faster than a list.

When an if -statement has an and -condition in it, the second condition is not valuated if it doesn't effect the value of the statement. In other words, if you can figure out which statement is more likely to fail and you valuate it first, you save some time by failing faster.
With list appends, you save time by pre-allocating the append to its own variable:

m = []
C = []
app_m = m.append
app_C = C.append
    for i in range(r):
        if i % 3 == 0:
            app_m(i)
            app_C(i)

If you haven't tried profiling your code to figure out what takes time, I would suggest doing that.

Script to evaluate point 1-3:

import timeit
from collections import deque

# for range
r = 20000
# timeit n times
ntimes = 500

def dicts():
    d = {"m": [], "C": []}
    for i in range(r):
        if i % 3 == 0:
            d["m"].append(i)
            d["C"].append(i)
    return d

def lists():
    m = []
    C = []
    for i in range(r):
        if i % 3 == 0:
            m.append(i)
            C.append(i)
    return {"m": m, "C": C}

def deques():
    m = deque(maxlen=r)
    C = deque(maxlen=r)
    for i in range(r):
        if i % 3 == 0:
            m.append(i)
            C.append(i)
    return {"m": m, "C": C}

t1 = timeit.timeit(dicts, number=ntimes)
t2 = timeit.timeit(lists, number=ntimes)
t3 = timeit.timeit(deques, number=ntimes)

print("dicsts", t1)
print("lists ", t2)
print("deques", t3)

def order1():
    m = []
    C = []
    for i in range(r):
        if i % 2 == 0 and i % 5 == 0:
            m.append(i)
            C.append(i)
    return {"m": m, "C": C}

def order2():
    m = []
    C = []
    for i in range(r):
        if i % 5 == 0 and i % 2 == 0:
            m.append(i)
            C.append(i)
    return {"m": m, "C": C}

t1 = timeit.timeit(order1, number=ntimes)
t2 = timeit.timeit(order2, number=ntimes)

print("order1", t1)
print("order2", t2)

def listsappend():
    m = []
    C = []
    app_m = m.append
    app_C = C.append
    for i in range(r):
        if i % 3 == 0:
            app_m(i)
            app_C(i)
    return {"m": m, "C": C}

t1 = timeit.timeit(lists, number=ntimes)
t2 = timeit.timeit(listsappend, number=ntimes)

print("normal", t1)
print("predef", t2)

I tested timing preallocating .append to a variable and i got a saving of 2μs over 81 million appends, not sure if that would save that much time unfortunately — Tom McLean, May 24 '21 at 11:47

Optimizing python code monte carlo simulation

1 Answers1