Check if subprocess is still running while reading its output line by line

Question

I am starting a subprocess in Python and trying to read each line of output. Unfortunately I can't find a good way of testing if my processes is still alive. The standard method seems to be checking poll(), but that seems to always return None. Here is my code.

proc = Popen(sys.argv[1:], stdin=PIPE, stdout=PIPE, stderr=STDOUT)

while True:
    process_line(proc.stdout.readline().decode('utf-8'))

    if not proc.poll():
        break

for line in proc.communicate()[0].splitlines():
    process_line(line.decode('utf-8'))

I've also tried using os.kill(proc.pid, 0), which works for non-spawned processes, but it seems that Python keeps a handle on processes it starts so os.kill(proc.pid, 0) always returns.

What am I doing wrong?

You opened a pipe to the process's stdin. Is it waiting to read data? — Chad S., Mar 07 '16 at 22:25
No the process only outputs data. In the future I want to add support for both directions, which is why I have that in there. — Kris Harper, Mar 07 '16 at 22:26

score 3 · Accepted Answer · answered Mar 07 '16 at 22:48

3

to process subprocess output line by line, try this:

p = subprocess.Popen(cmd, stdout=subprocess.PIPE, bufsize=1)
while p.poll() is None:
    line = p.stdout.readline()
    ...do something with the line here

notice I set the buffer size to 1.

also, os.kill(proc.pid, 0) is not needed. Just call .kill() on the subprocess you spawned. You can also call .wait() to wait for termination instead of immediately killing the process.

answered Mar 07 '16 at 22:48

Corey Goldberg

59,062
28
129
143

1

This seems to work. Thanks. – Kris Harper Mar 07 '16 at 23:39
@KrisHarper: there is *no need* to check whether the process is alive if you want to capture its output line by line. – jfs Mar 08 '16 at 18:51
@J.F.Sebastian Hmm so in your Python 3 answer here http://stackoverflow.com/a/17698359/817630, there is no risk of the process ending while lines are still being sent to stdout? Before I was using this answer http://stackoverflow.com/a/2716032/817630 to the same question, but I see now that it recommends yours instead. – Kris Harper Mar 08 '16 at 21:47
@KrisHarper if the process is dead then (obviously) it doesn't produce any output and — as soon as already buffered data is read — the loop ends. – jfs Mar 08 '16 at 21:53

Check if subprocess is still running while reading its output line by line

1 Answers1