Extra processes that linger

Question

My class for handling processes, somewhat shortened for simplicity, looks like this:

class LocalEngine:

    def __init__(self):
        name_to_pid = {}

    def run_instance(self, name):
        args = ... # Computing args for Popen from arguments
        with open(f"out-{name}", 'a') as out, open(f"err-{name}", 'a') as err:
            pid = \
                subprocess.Popen(args, stdout=out, stderr=err, shell=False).pid
            print(f"Created process {pid}")
            self.name_to_pid[name] = pid

    def kill_instance(self, name):
        pid = self.name_to_pid[name]
        print(f"Killing process {pid}")
        psutil.Process(pid).kill()

There is only one object engine of type LocalEngine and all processes are created using engine.run_instance. At the end, all processes are killed using engine.kill_instance.

Here is the relevant part of the main script's output (under Ubuntu):

Created process 9676
Created process 9703
Killing process 9703
Killing process 9676

However, this is what ps auxfww outputs after all processes are created:

root     27576  0.0  0.0    896    80 ?        S    10:33   0:00  \_ /init
meir     27577  0.0  0.0  10688  5944 pts/2    Ss   10:33   0:00      \_ -bash
meir      9667  2.2  0.0  24296 16956 pts/2    S+   15:26   0:00          \_ python -m examples.agent_assignment.run_server
meir      9668  0.3  0.0 392712 16732 pts/2    Sl+  15:26   0:00              \_ python -m examples.agent_assignment.run_server
meir      9676  1.9  0.0  24016 16340 pts/2    S+   15:26   0:00              \_ python -m src.run_backup 127.0.1.1 57785 server-1
meir      9677  3.7  0.0 392784 14648 pts/2    Sl+  15:26   0:00              |   \_ python -m src.run_backup 127.0.1.1 57785 server-1
meir      9703  8.0  0.0  28364 21084 pts/2    S+   15:26   0:00              \_ python -m examples.agent_assignment.run_client 127.0.1.1 57785 client-1 2
meir      9704 10.0  0.0 765568 18912 pts/2    Sl+  15:26   0:00                  \_ python -m examples.agent_assignment.run_client 127.0.1.1 57785 client-1 2

run_server is the main script. The two calls to Popen run the scripts named run_backup and run_client, respectively. The above output of ps shows that, for some reason, there are two processes for each call of Popen. Since I pass shell=False to Popen, I would not expect that to happen. Here is the output of ps after the processes are killed:

root     27575  0.0  0.0    896    80 ?        Ss   10:33   0:00 /init
root     27576  0.0  0.0    896    80 ?        S    10:33   0:00  \_ /init
meir     27577  0.0  0.0  10688  5944 pts/2    Ss+  10:33   0:00      \_ -bash
meir      9677  0.1  0.0 466516 14736 pts/2    Sl   15:26   0:00      \_ python -m src.run_backup 127.0.1.1 57785 server-1
meir      9704  0.0  0.0 749176 21088 pts/2    Sl   15:26   0:00      \_ python -m examples.agent_assignment.run_client 127.0.1.1 57785 client-1 2

What are these extra processes that linger? How do I prevent them from being created?

EDIT AFTER FEEDBACK:

Based on the reply by @AKX I changed the kill_instance method. I cannot switch to storing process objects instead of pid's due to the requirement of unpickling by another process.

    def kill_instance(self, name):
        # Process tree with full commands: ps auxfww
        pid = self.name_to_pid[name]
        print(f"Terminating process {pid}")
        proc = psutil.Process(pid)
        proc.terminate()
        print(f"Waiting for process {proc} to terminate")
        try:
            proc.wait(timeout=10)
        except subprocess.TimeoutExpired:
            print(f"Process {pid} did not terminate in time, killing it")
            proc.kill()

The exception is not triggered, but there are still processes lingering as in the original question...

UPDATE:

Got it! The second process appears because of starting multiprocessing manager. Now the question is how to terminate both the process and the manager...

Answer 1

I think using kill() (the SIGKILL signal) is a bit problematic; you should try to terminate() (SIGTERM) the processes first.

Also, it'd be better to do that via the Popen object, so Python's side of the Popen has time to deal with things:

import subprocess


class LocalEngine:
    def __init__(self):
        self.name_to_proc: dict[str, subprocess.Popen] = {}

    def run_instance(self, name):
        args = ...  # Computing args for Popen from arguments
        with open(f"out-{name}", 'a') as out, open(f"err-{name}", 'a') as err:
            proc = subprocess.Popen(args, stdout=out, stderr=err, shell=False)
            print(f"Created process {proc.pid}")
            self.name_to_proc[name] = proc

    def kill_instance(self, name, timeout=10):
        proc = self.name_to_proc[name]
        print(f"Killing process {proc}")
        proc.terminate()
        print(f"Waiting for process {proc} to terminate")
        try:
            proc.wait(timeout=timeout)
        except subprocess.TimeoutExpired:
            print(f"Process {proc} did not terminate in time, killing it")
            proc.kill()