Multiprocessing and matlibplot, non-blocking plots

Question

I know there are quite a few other questions on this topic and i have read/experimented with the solutions but none of them quite do what i want or dont work ...

What im trying to do:

I would like to create a plot using matplotlib with the plot launched in a new process such that it is open and responsive while the main thread is running, additionally when the main thread exits not to kill the child processes. I would like something portable if possible hence i have avoided fork so far as my impression is that it isn't (correct?). I have tried the show(block=False) in matplotlib. If there isn't anything else im happy to accept that but preferably the main thread exits and the plots are still sitting there.

My best attempt so far:

This code is modified from Python close children when closing main process

import logging, signal, sys, time
import multiprocessing as mp 
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid.axislines import SubplotZero

class AddProcessNameFilter(logging.Filter):
    """Add missing on Python 2.4 `record.processName` attribute."""
    def filter(self, r):
        r.processName = getattr(r, 'processName', mp.current_process().name)
        return logging.Filter.filter(self, r)

def print_dot(plt):
    signal.signal(signal.SIGTERM, signal.SIG_IGN)
    while True:
        mp.get_logger().info(".")
        plt.show(block=True)

def main():
    logger = mp.log_to_stderr()
    logger.setLevel(logging.INFO)
    logger.addFilter(AddProcessNameFilter()) # fix logging records

    fig = plt.figure()
    ax = fig.add_subplot(111)
    ax.plot([0, 1, 2, 3], [1, 1, 3, 2],'ro')

    # create daemonic child processes
    p = mp.Process(target=print_dot, args=(plt,))
    p.daemon = True
    p.start()    

if __name__=="__main__":
    mp.freeze_support()
    main()

What happens to this code ...

The plot isn't launched additionally the main thread doesn't exit it just waits on join. When interrupted I get the following error as well ...

X Error of failed request:  BadIDChoice (invalid resource ID chosen for this connection)
  Major opcode of failed request:  53 (X_CreatePixmap)
  Resource id in failed request:  0x4400017
  Serial number of failed request:  344
  Current serial number in output stream:  352

Not really sure what to make of that. Any ideas? Thanks in advance :D

PS: Ubuntu 12.04 ... but i don't know if thats particularly relevant in this case ...

do it the other way around, keep the plot on the main thread, computations in another process. And you should also look into embedding matplotlib in to one of the toolkits (which have already done a lot of the portability problem for you) and have proper threading support. — tacaswell, Jan 15 '14 at 03:59
hmmm ... that could be an option, but i would need to wait on the data with p.join()? the plots wouldn't be responsive would they? (ie save etc). ill have a look at the toolkits :D — da_steve101, Jan 15 '14 at 04:18
use the queues that are in `multiprocess` + `signals` to poll them at intervals. — tacaswell, Jan 15 '14 at 04:40
Possible duplicate of [Python IPC with matplotlib](https://stackoverflow.com/questions/48603249/python-ipc-with-matplotlib) — csandy83, Mar 18 '18 at 20:03

score 0 · Answer 1 · answered Jun 02 '14 at 01:00

I use the following code. It's similar to yours, but joins old processes as the function is supposed to be called many times. You can close the main process and the plot won't be closed.

There's no guarantee that the plot will be shown just after the p.start is called. But this would be very unusual and happened to me just few times. And in these times the window is shown either if plot is called again or if I call p.is_alive().

    import multiprocessing as mp
    import sys
    def foo(*arg):
        import matplotlib.pyplot as plt
        plt.ioff()    
        plt.plot(*arg)
        plt.show()

    if __name__ == '__main__':
        mp.freeze_support()
        arg=(range(10),)
        global pl

        #create the process list, if it does not exists
        try:
            pl
        except NameError:
            pl = []    

        #Join old ones to keep zombie process list small
        for i in range(len(pl)):
            if not (pl[i] is None):
                if not pl[i].is_alive():
                    pl.pop(i).join()


        #create plot process
        p = mp.Process(target=foo, args=arg)
        p.start()
        pl.append(p)

score 0 · Answer 2 · answered Jul 26 '20 at 00:07

I recommend a solution from https://stackoverflow.com/a/62354049/13995577. Works well for me. It is a bit of a hack though, but relatively small.

import multiprocessing as mp
def my_plot_routine(q):
    ...
    q.put('Done, see plot window')
    plt.show()
    return

class EverLastingProcess(mp.Process):
    def join(self, *args, **kwargs):
        pass

    def __del__(self):
        pass

if __name__ == '__main__':
    # create plot process
    mp.set_start_method('spawn')
    q = mp.Queue()
    p = EverLastingProcess(target=my_plot_routine, args=(q,))
    p.start()
    print(q.get())

Multiprocessing and matlibplot, non-blocking plots

2 Answers2