Capture continous output of external program

Question

I' m trying to capture the output of AtomicParsley which flows as parsley runs like

Started writing to temp file.
 Progress: >0%-----------------------------------------------------------------------------|
 Progress: =>1%----------------------------------------------------------------------------|
 Progress: ==>2%---------------------------------------------------------------------------|
    ...
 Progress: ======================================================================>95%--|
 Progress: =======================================================================>96%--|
 Progress: ========================================================================>97%--|
 Progress: =========================================================================>98%--|
 Progress: ==========================================================================>99%--|
 Progress: ===========================================================================>100%|
Finished writing to temp file.

but it all gets printed at once when it is finished. The code I have is:

process = subprocess.Popen([atomicparams], shell=True, stdout=PIPE)
for line in iter(process.stdout.readline, ""):
    print line,

I've read all similar answers but they don't seem to fit to what I need (I need the printed lines to feed a progress bar). Could someone help?

Does this question helps ? https://stackoverflow.com/questions/4760215/running-shell-command-and-capturing-the-output I especially think about vartec's answer which seem to suit you — Diane M, May 08 '19 at 10:14
Tried vartec's solution (changed yield with print) but still prints all "percentage" lines at once when parsley is finished. — Filippos, May 08 '19 at 12:08

score 1 · Accepted Answer · answered May 08 '19 at 12:39

It seems your program hangs because AtomicParsley never returns a line, but instead use escape codes to erase the same line over and over and get it reprinted for dynamic output. In order to reproduce this in the terminal, you could print it char by char once available to the parent process.

import subprocess
import sys

p = subprocess.Popen([atomicparams], stdout=subprocess.PIPE)
while(True):
    # returns None while subprocess is running
    retcode = p.poll() 
    sys.stdout.buffer.write(p.stdout.read(1))
    sys.stdout.buffer.flush()
    if retcode is not None:
        break

Capture continous output of external program

1 Answers1