How to profile a script which uses python's multiprocessing

Question

I have a script file which uses the function parallel_map. ( source code of qutip.parallel.parallel_map) from a package QuTiP . As one would see on clicking the source code for the function, it uses the multiprocess module of python. I looked at answers of serial version of this question. I decided upon Snakeviz on reading zaxiliu's solution. But naively trying it on my code fails. So what must I do to profile my code? My heart is not set on Snakeviz. I don't mind using any other graphical tool.

Answer 1

Doesn't satisfy the question requirements fully, but will work if nothing else is available

Try using serial_map instead of parallel_map from the same module. Replace (or better yet comment out) the line

from qutip.parallel import parallel_map

with

from qutip.parallel import serial_map

Now you have a serial implementation of code. This can be profiled using those described in serial version of your question. After this (assuming you go ahead with Snakeviz)

• make the profile file

  python -m cProfile -o program.prof my_program.py

• Run Snakeviz on the profile file generated in the previous step

  snakeviz program.prof