not able to terminate the process in multiprocessing python (linux)

Question

I am new to python and using multiprocessing, I am starting one process and calling one shell script through this process. After terminating this process shell script keeps running in the background, how do I kill it, please help.

python script(test.py)

#!/usr/bin/python
import time
import os
import sys 
import multiprocessing

# test process 
def test_py_process():
    os.system("./test.sh")
    return


p=multiprocessing.Process(target=test_py_process)
p.start()
print 'STARTED:', p, p.is_alive()

time.sleep(10)

p.terminate()
print 'TERMINATED:', p, p.is_alive()

shell script (test.sh)

#!/bin/bash

for i in {1..100}
do
    sleep 1
    echo "Welcome $i times"
done

shmee · Answer 1 · 2018-06-18T11:12:24.460

The reason is that the child process that is spawned by the os.system call spawns a child process itself. As explained in the multiprocessing docs descendant processes of the process will not be terminated – they will simply become orphaned. So. p.terminate() kills the process you created, but the OS process (/bin/bash ./test.sh) simply gets assigned to the system's scheduler process and continues executing.

You could use subprocess.Popen instead:

import time
from subprocess import Popen

if __name__ == '__main__':
    p = Popen("./test.sh")
    print 'STARTED:', p, p.poll()

    time.sleep(10)

    p.kill()
    print 'TERMINATED:', p, p.poll()

Edit: @Florian Brucker beat me to it. He deserves the credit for answering the question first. Still keeping this answer for the alternate approach using subprocess, which is recommended over os.system() in the documentation for os.system() itself.

score 0 · Answer 2 · answered Jun 18 '18 at 10:59

os.system runs the given command in a separate process. Therefore, you have three processes:

The main process in which your script runs
The process in which test_py_processes runs
The process in which the bash script runs

Process 2 is a child process of process 1, and process 3 is a child of process 1.

When you call Process.terminate from within process 1 this will send the SIGTERM signal to process two. That process will then terminate. However, the SIGTERM signal is not automatically propagated to the child processes of process 2! This means that process 3 is not notified when process 2 exits and hence keeps on running as a child of the init process.

The best way to terminate process 3 depends on your actual problem setting, see this SO thread for some suggestions.

not able to terminate the process in multiprocessing python (linux)

python script(test.py)

shell script (test.sh)

2 Answers2