2

I use subprocess.Popen to spawn a new child process - in this case git clone.

The issue is that git clone itself spawns subprocesses, and when I try to kill them using Popen.kill() only the parent (git clone) gets killed, but not its children.

An example of a child is:

79191 /usr/lib/git-core/git fetch-pack --stateless-rpc --stdin --lock-pack --include-tag --thin --cloning --depth=1 https://example.com/scm/adoha/adoha_digit_interpretation.git/

How can I kill all of the processes - git clone and its children?

NB: I've thought about placing the processes in their own process group, but then the main process gets killed as well.

    # execute a child process using os.execvp()
    p = subprocess.Popen(shlex.split(f'git clone --bare --depth=1 -v \'{url}\' \'{temp_dir}\''),
                         stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
    try:
        ret_code = p.wait(timeout)
    except subprocess.TimeoutExpired as exc:
        p.kill()
        shutil.rmtree(temp_dir)
        raise common.exc.WatchdogException(f'Failed to clone repository: Timeout.'
                                           f'\n{timeout=}\n{url=}') from exc
Shuzheng
  • 11,288
  • 20
  • 88
  • 186

1 Answers1

1

You can kill a process and its children with this snippet:

import psutil


def kill_process_and_children(pid: int, sig: int = 15):
    try:
        proc = psutil.Process(pid)
    except psutil.NoSuchProcess as e:
        # Maybe log something here
        return

    for child_process in proc.children(recursive=True):
        child_process.send_signal(sig)

    proc.send_signal(sig)

kill_process_and_children(p.pid)
RobinFrcd
  • 4,439
  • 4
  • 25
  • 49
  • Thank you, Robin. Could this result in a race condition, where a process spawns a child after `proc_ children()` has been called, but before it has been killed? What about starting a process through a shell - will the shell not take care of killing all of the children then (they are in the same process group)? – Shuzheng Oct 03 '20 at 10:17
  • If the process spawns after `proc.children`, it won't be killed, yes. When using `shell=True` (not recommended: https://docs.python.org/3.8/library/subprocess.html#security-considerations) the parent process will be then shell, so killing the shell will have its children go defunct (not instant killed). See: https://stackoverflow.com/a/41961462/5692012 – RobinFrcd Oct 03 '20 at 10:43
  • Do you know how the issue "*If the process spawns after proc.children, it won't be killed, yes.*" can be fixed? – Shuzheng Oct 03 '20 at 15:04