How to pass exception from one process to another?

Question

I want to raise an exception in the upload function if the running status in stop function is 'stopped'. This doesn't seem to work. I am using Pipe to pass the exception. What is wrong?

def upload(instances, u1):
    for instance in instance:
        try:
            u1.recv()
            #do_something
        except:
            #do_something_else
            continue         

def stop(instances, s1):
    for instance in instances:
        RunningStatus = instance[4]
        if RunningStatus.lower() == 'stopped'.lower():
            s1.send(Exception) # I want to raise exception in upload function 
                               # from here

if __name__ == '__main__':
    s1, u1 = multiprocessing.Pipe()
    s = multiprocessing.Process(target = stop, args = (instances, s1,))
    u = multiprocessing.Process(target = upload, args = (instances, u1))
    s.start()
    u.start()
    u.join()

I would try to simplify your problem to learn how to work with exceptions in python. If you're still having trouble, try to create the most http://stackoverflow.com/help/mcve example of your problem. Just asking how to pass an exception "correctly" is too vague. — PerryC, Dec 08 '16 at 18:35
@PerryC I have changed the wording to make it more understandable. — gns, Dec 08 '16 at 18:57
You can't directly. However you could use a shared `multiprocessing.Value` to allow one process to communicate with the other. The `Value` could used as a flag to indicate when an exception has occurred. If you want additional information, then you need to convey more information between them (perhaps using a `Queue`). See the [Sharing state between processes](https://docs.python.org/2/library/multiprocessing.html#sharing-state-between-processes) section of the documentation. — martineau, Dec 08 '16 at 19:19
Does this answer your question? [How to pass stacktrace between processes in Python?](https://stackoverflow.com/questions/56481100/how-to-pass-stacktrace-between-processes-in-python) — Darkonaut, Apr 02 '21 at 21:05

score 2 · Accepted Answer · answered Dec 08 '16 at 19:56

Here's an example of how you could send Exception objects from one process to another. I also tried sending complete exception information (as returned by sys.exc_info), but, reasonably enough, this fails. One could always format the traceback info to a string and send that instead (see the traceback module).

Tested on Ubuntu 14.04 Python 2.7, 3.4 (Ubuntu-provided), and 3.5 (Continuum).

from __future__ import print_function

import sys
import multiprocessing
import time

def upload(u1):
    i=0
    try:
        while True:
            print('>upload>',i)
            i+=1
            if u1.poll():
                # tvt = u1.recv()
                # raise tvt[0], tvt[1], tvt[2] # Python 2.7
                e = u1.recv()
                raise e
            time.sleep(0.1)
    except Exception as e:
        print('Exception caught:',e)
        print('exiting')

def stop(s1):
    try:
        while True:
            for j in range(100,110):
                time.sleep(0.1)
                if 105==j:
                    raise RuntimeError("oh dear at j={}".format(j))
    except Exception as e:
        # tvt = sys.exc_info()
        # s1.send(tvt) # doesn't work; tracebacks are not pickle'able
        s1.send(e)

if __name__ == '__main__':
    s1, u1 = multiprocessing.Pipe()
    s = multiprocessing.Process(target = stop, args = (s1,))
    u = multiprocessing.Process(target = upload, args = (u1,))
    s.start()
    u.start()
    u.join()

Output:

>upload> 0
>upload> 1
>upload> 2
>upload> 3
>upload> 4
>upload> 5
>upload> 6
Exception caught: oh dear at j=105
exiting

`Pipe`s seem like a good choice for this sort of interprocess communications. To format the traceback info to a string, one can use `traceback.print_exc()` with an in-memory bytes buffer provided by an `io.BytesIO()` object as a `file=` argument to receive the data from it to be sent. — martineau, Dec 09 '16 at 02:32
This worked perfectly for my requirements, even though I was working on Windows. Thanks. — gns, Dec 09 '16 at 16:35
But how to catch the exception when there is no while/true or any other loop? — parovelb, Apr 06 '21 at 07:09

score 0 · Answer 2 · edited Oct 23 '19 at 03:15

You might want to look into using an Event object instead of a Pipe and Exception combination. By using an Event primitive to share data between your processes you could have the Upload function watch the event and trigger your logic when the stop function finds an issue.

Based on your example I'm assuming that each instance object in instances is an array, so by extending that array with an additional element you can have a unique event for each instance.

def upload(instances, u1):
    for instance in instance:
        if instance[5].is_set(): # events only return TRUE if they are set
            #Exception logic would go here

    else: 
        u1.recv()
        #do_something


def stop(instances, s1):
    for instance in instances:
        RunningStatus = instance[4]
        if RunningStatus.lower() == 'stopped'.lower():
            instance[5].set() # Set the event to TRUE 

if __name__ == '__main__':
    for instance in instances:
        instance[5] = multiprocessing.Event() # create a new event semaphore 
    s = multiprocessing.Process(target = stop, args = (instances, s1,))
    u = multiprocessing.Process(target = upload, args = (instances, u1))
    s.start()
    u.start()
    u.join()

The full list of multiprocessing primitives can be found over here: https://docs.python.org/2/library/multiprocessing.html#synchronization-primitives

Here's some good examples of various uses for primitives other multiprocessing objects: https://pymotw.com/2/multiprocessing/communication.html

How to pass exception from one process to another?

2 Answers2

Linked