Trying to sum a Taylor series with Python Multiprocessing

Question

Hi all I made this function that gives the sum of the first n terms of the Taylor expansion for arctan: I'm using mpmath module with this, mpf is the arbitrary precision float type.

def atantaylor(x,n):
#Taylor series expansion of arctan about 0 for n terms, evaluated at x
sum = 0
xt = x
xsq = x**2
for j in range(n):  
    nterm = ((-1)**j * xt) /(2.0*j+1)
    xt = xt * xsq
    sum += nterm
return sum

I want to make this run on a few cores so I made another function that sums from term n to n+cs, ideally so I can run a few of these in parallel to make it faster:

def taylor1(x,n,cs):
#Sum of Taylor series of atan from n to n + cs - 1
sum = mpf(0)
#Find term n
xt = mpf(x **(2*n + 1))
xsq = mpf(x**2)
#Loop from n to n + cs - 1
for j in range(n, n+cs):        
    nterm = ((-1)**j * xt) /(2.0*j+1)
    xt = xt * xsq
    sum += nterm
    print "term %d is %r" % (j, nterm)
print sum
return sum

The idea here is that I can run a few processes with intervals [0, cs] [cs, cs*2] [cs*2, cs*3].

I'm pretty new to multiprocessing and I have the following code mimicked from this tutorial here

def mp_atan(x, n, nprocs):
    #nprocs must be a factor of n so each worker can receive the same number of tasks that are integer value

    if n % nprocs != 0:
        print "please give an n that is a multiple of nprocs"
        return 0

    def worker(x, n, chunksize, out_q):
        a = tan_n(x, n, chunksize)
        out_q.put([a])

    out_q = mp.Queue()
    chunksize = int(n / nprocs)
    procs = []

    for i in range(nprocs):
        p = mp.Process(
                target = worker,
                args=(x, chunksize * i, chunksize, out_q,)
                )
        procs.append(p)
        p.start()

    #sum results
    sum = 0
    for i in range(nprocs):
        sum += out_q.get()

    #wait for all workers to finish
    for p in procs:
        p.join()

    return sum

I'm getting an EOFError and "pickle.PicklingError: Can't pickle : it's not found as mc.worker"

Is there anyway to get this up and running?

I don't have `mpf` but here's something obvious to test: what happens if you fire up an interactive python, `import mpmath, pickle`, and `print pickle.dumps(mpmath.mpf(0))`? If you get a PicklingError, that's the problem: mpf objects are not pickle-able. (The multiprocessing module and its `Queue` type dump and load objects to transfer them between processes.) — torek, May 26 '13 at 00:06
no error my friend. Returned a bunch of strings: '>>> print pickle.dumps(mpmath.mpf(0)) ccopy_reg _reconstructor p0 (cmpmath.ctx_mp_python mpf p1 c__builtin__ object p2 Ntp3 Rp4 (I0 S'0' p5 I0 L0L tp6 b. >>>' — dedalux, May 26 '13 at 01:17
OK, that's not the problem then! Not sure what is. It's not something else that seems odd above: `out_q.put([a])` drops a list-ified value into the output queue, but `sum += out_q.get()` seems to expect the original value, without the enclosing list. But that would not give the error you're seeing. — torek, May 26 '13 at 01:43

score 0 · Answer 1 · edited May 23 '17 at 12:30

For arbitrary function passing problems ->

I find myself fixing 99% of problems by creating deep copies of functions being passed.

See this post to make a function deep copy: How can I make a deepcopy of a function in Python?

If memory is not a bottleneck, and you just want stuff to work, it might be worth a shot.

import types
def copy_func(f, name=None):
    return types.FunctionType(f.func_code, f.func_globals, name or f.func_name,
        f.func_defaults, f.func_closure)

So now you can try:

for i in range(nprocs):

    workercopy = copy_func(worker)

    p = mp.Process(
            target = workercopy,
            args=(x, chunksize * i, chunksize, out_q,)
            )
    procs.append(p)
    p.start()

Trying to sum a Taylor series with Python Multiprocessing

1 Answers1