5

I have a subprocess which I open, which calls other processes.

I use os.killpg(os.getpgid(subOut.pid), signal.SIGTERM) to kill the entire group, but this kills the python script as well. Even when I call a python script with os.killpg from a second python script, this kills the second script as well. Is there a way to make os.killpg not stop the script?

Another solution would be to individually kill every child 1process. However, even using 

p = psutil.Process(subOut.pid)
child_pid = p.children(recursive=True)
for pid in child_pid:
    os.kill(pid.pid, signal.SIGTERM)

does not correctly give me all the pids of the children.

And you know what they say... don't kill the script that calls you...

user3632833
  • 71
  • 1
  • 2
  • 4

4 Answers4

10

A bit late to answer, but since google took me here while looking for a related problem: the reason your script gets killed is because its children will, by default, inherit its group id. But you can tell subprocess.Popen to create a new process group for your subprocess. Though it's a bit tricky: you have to pass in os.setpgrp for the preexec_fn parameter. This will call setpgrp (without any arguments) in the newly created (forked) process (before that does the exec) which will set the gid of the new process to the pid of the new process (thus creating a new group). The documentation mentions that it can deadlock in multi-threaded code. As an alternative, you can use start_new_session=True, but that would create not only a new process group but a new session. (And that would mean that if you close your terminal session while your script is running, the children would not be terminated. It may or may not be a problem.)

As a side note, if you are on windows, you can simply pass subprocess.CREATE_NEW_PROCESS_GROUP in the creationflag parameter.

Here is what it looks like in detail:

subOut = subprocess.Popen(['your', 'subprocess', ...], preexec_fn=os.setpgrp)

# when it's time to kill
os.killpg(os.getpgid(subOut.pid), signal.SIGTERM)
atleta
  • 321
  • 3
  • 6
  • 1
    The original version of the answer referred to the `os.setgrp` function (which doesn't exist) instead of the correct `os.setpgrp`. Thanks to @mowiat for pointing out. – atleta Nov 11 '20 at 13:11
  • this really saves me a lot of time, thanks! – Junning Huang Oct 19 '22 at 08:42
0

Create a process group having all the immediate children of the called process as follows:

p1 = subprocess.Popen(cmd1)
os.setpgrp(p1.pid, 0) #It will create process group with id same as p1.pid
p2 = subprocess.Popen(cmd2)
os.setpgrp(p2.pid, os.getpgid(p1.pid))

pn = subprocess.Popen(cmdn)
os.setpgrp(pn.pid, os.getpgid(p1.pid))

#Kill all the children and their process tree using following command
os.killpg(os.getpgid(p1.pid), signal.SIGKILL)

It will kill whole process tree except its own process.

Vivek Agrawal
  • 113
  • 11
0

atleta's answer above worked for me but the preexec_fn argument in the call to Popen should be setpgrp, rather than setgrp:

subOut = subprocess.Popen(['your', 'subprocess', ...], preexec_fn=os.setpgrp)

I'm posting this as an answer instead of a comment on atleta's answer because I don't have comment privileges yet.

mowiat
  • 21
  • 3
0

Easy way is to set the parent process to ignore the signal before sending it.

# Tell this (parent) process to ignore the signal
old_handler = signal.signal(sig, signal.SIG_IGN)

# Send the signal to our process group and
# wait for them all to exit.
os.killpg(os.getpgid(0), sig)
while os.wait() != -1:
   pass

# Restore the handler
signal.signal(sig, old_handler)
Douglas Ferreira
  • 707
  • 2
  • 5
  • 22
Steve Campbell
  • 51
  • 2