Reading stdout after terminating a process

Question

I would like to launch a process, let it run for some time and then read its output (the last output is ok, I don't need everything). I tried to use the following code:

def connect(interface):
    proc = subprocess.Popen(['mycommand', interface], stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
    time.sleep(10)
    proc.terminate()
    output, err = proc.communicate()
    print(output)

Unfortunately, every time it gets stuck when reading the output. I also tried to use proc.read() instead of communicate() but it didn't solve the problem.

What's the best way to handle output in this case?

Many thanks in advance!

Maybe see https://stackoverflow.com/questions/375427/non-blocking-read-on-a-subprocess-pipe-in-python — tripleee, Sep 11 '18 at 08:35

score 0 · Accepted Answer · answered Sep 12 '18 at 06:17

After some research, I found that the issue came from the buffer. As indicated in the subprocess module documentation:

This will deadlock when using stdout=PIPE or stderr=PIPE and the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data. Use Popen.communicate() when using pipes to avoid that.

There are two solutions:

Use the bufsize argument to set the buffer to a size large enought to store all the output generated during the time you wait.
Use readline() to read the output and flush the buffer during the waiting time. If you don't need it just discard the output.

I choose the second approach ; my code is as follow:

def connect(interface):
    stdout = []
    timeout = 60
    start_time = time.time()
    proc = subprocess.Popen(['mycommand', interface], stdout=subprocess.PIPE, stderr=subprocess.PIPE, bufsize=1, universal_newlines=True)
    while time.time() < start_time + timeout:
        line = proc.stdout.readline()
        stdout.append(line)
    proc.terminate()
    print(output)

Reading stdout after terminating a process

1 Answers1