Terminate child process on subprocess.TimeoutExpired

Question

I have the following snippet of code:

    def terminal_command(command, timeout=5*60):
        """Executes a terminal command."""
        cmd = command.split(" ")

        timer = time.strftime('%Hh %Mm %Ss', time.gmtime(timeout))

        proc = None

        try:
            proc = subprocess.run(cmd, timeout=timeout, capture_output=True)
        except subprocess.TimeoutExpired:
            print("Timeout")
            proc.terminate()
            reason = "timeout"
            stdout = b'error'
            stderr = b'error'

        if proc != None:
            # Finished!

            stdout = proc.stdout
            stderr = proc.stderr
            reason = "finished"

        return stdout.decode('utf-8').strip(), stderr.decode('utf-8').strip(), reason

I ran a command which takes significantly longer than 5 minutes. In this instance, subprocess.run raises an exception, but proc is now None so I cannot use proc.terminate(). When the code terminates, as has been well documented elsewhere, the child process continues to run. I would like to terminate it.

Is there any way to terminate a subprocess on a TimeoutExpired, whilst redirecting output? I am on a Linux system so am open to requiring Popen but ideally I would like this to be cross-platform.

The [documentation](https://docs.python.org/3/library/subprocess.html#subprocess.run) for `subprocess.run` says "If the timeout expires, the child process will be killed and waited for." This sounds like it terminates the child process for you. — rchome, Jan 05 '22 at 04:12
Then this is a bug - certainly when the above code is run, the child process it runs writes to a file. When `subprocess.run` times out, the file is still being written to - I can run `tail -f outfile` and observe output still being written, even after the process is apparently killed. — Seabody, Jan 05 '22 at 20:33
@rchome the Doc talks about the child process not child processes, child process in the Doc means the parent process you run inside run(), which is a child process of run(). — Badr Elmers, Jan 19 '22 at 20:18

score 0 · Accepted Answer · answered May 06 '22 at 03:25

Four months later: I got it.

The core issue appears to be that using os.kill with signal.SIGKILL doesn't properly kill the process.

Modifying my code to the following works.

    def custom_terminal_command(self, command, timeout=5*60, cwd=None):
        with subprocess.Popen(command.split(" "), preexec_fn=os.setsid) as process:
            wd = os.getcwd()
            try:
                if cwd is not None:
                    # Man fuck linux
                    for d in cwd.split("/"):
                        os.chdir(d)
                stdout, stderr = process.communicate(None, timeout=timeout)
            except subprocess.TimeoutExpired as exc:
                import signal

                os.killpg(os.getpgid(process.pid), signal.SIGTERM)

                try:
                    import msvcrt
                except ModuleNotFoundError:
                    _mswindows = False
                else:
                    _mswindows = True

                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                reason = 'timeout'
                stdout, stderr = process.communicate()
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                reason = 'other'
                stdout, stderr = process.communicate()
                raise
            else:
                reason = 'finished'
            finally:
                os.chdir(wd)

            try:
                return stdout.decode('utf-8').strip(), stderr.decode('utf-8').strip(), reason
            except AttributeError:
                try:
                    return stdout.strip(), stderr.strip(), reason
                except AttributeError:
                    return stdout, stderr, reason

See the following SO post for a short discussion: How to terminate a python subprocess launched with shell=True

Terminate child process on subprocess.TimeoutExpired

1 Answers1