Terminate subprocess running in thread on program exit

Question

Based on the accepted answer to this question: python-subprocess-callback-when-cmd-exits I am running a subprocess in a separate thread and after the completion of the subprocess a callable is executed. All good, but the problem is that even if running the thread as a daemon, the subprocess continues to run even after the program exits normally or it is killed by kill -9, Ctrl + C, etc...

Below is a very simplified example (runs on 2.7):

import threading
import subprocess
import time
import sys


def on_exit(pid):
    print 'Process with pid %s ended' % pid

def popen_with_callback(cmd):

    def run_in_thread(command):
        proc = subprocess.Popen(
            command,
            shell=False
        )
        proc.wait()
        on_exit(proc.pid)
        return

    thread = threading.Thread(target=run_in_thread, args=([cmd]))
    thread.daemon = True
    thread.start()
    return thread

if __name__ == '__main__':
    popen_with_callback(
        [
            "bash", 
            "-c",
            "for ((i=0;i<%s;i=i+1)); do echo $i; sleep 1; done" % sys.argv[1]
        ])
    time.sleep(5)
    print 'program ended'

If the main thread lasts longer than the subprocess everything is fine:

(venv)~/Desktop|➤➤ python testing_threads.py 3
> 0
> 1
> 2
> Process with pid 26303 ended
> program ended

If the main thread lasts less than the subprocess, the subprocess continues to run until it eventually hangs:

(venv)~/Desktop|➤➤ python testing_threads.py 8
> 0
> 1
> 2
> 3
> 4
> program ended
(venv)~/Desktop|➤➤ 5
> 6
> 7
# hanging from now on

How to terminate the subprocess if the main program is finished or killed? I tried to use atexit.register(os.kill(proc.pid, signal.SIGTERM)) just before proc.wait but it actually executes when the thread running the subprocess exits, not when the main thread exits.

I was also thinking of polling for the parent pid, but I am not sure how to implement it because of the proc.wait situation.

Ideal outcome would be:

(venv)~/Desktop|➤➤ python testing_threads.py 8
> 0
> 1
> 2
> 3
> 4
> program ended
> Process with pid 1234 ended

score 0 · Answer 1 · answered Dec 06 '16 at 13:26

Use Thread.join method, which blocks main thread until this thread exits:

if __name__ == '__main__':
    popen_with_callback(
        [
            "bash", 
            "-c",
            "for ((i=0;i<%s;i=i+1)); do echo $i; sleep 1; done" % sys.argv[1]
        ]).join()
    print 'program ended'

score 0 · Answer 2 · answered Nov 14 '22 at 16:11

I just got an ugly but effective method.
Just set a global variable to handle the proc=subprocess.Popen(),
and you can kill the proc whenever you like:

my_proc = None

def popen_with_callback(cmd):
  def run_in_thread(command):
    global my_proc
    proc = subprocess.Popen(command, shell=False)
    my_proc = proc
    proc.wait()
    on_exit(proc.pid)
    return
...

Then you can kill the proc wherever you want in the program. just do:

my_proc.kill()

Terminate subprocess running in thread on program exit

2 Answers2