2

Here's my subprocess call:

def myrun(cmd):
  p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
  stdout = []
    while True:
      line = p.stdout.readline()
      stdout.append(line)
      print line,
      if line == '' and p.poll() != None:
          break
  return ''.join(stdout)

When I run the 'cmd' normally, the output is usually something like:

some text...
some more text...
Do you want to continue [yes/no]? : y
more output...

But running the same 'cmd' with subprocess as shown above, my output is like this:

Do you want to continue [yes/no]? : y
some text...
some more text...
more output...

How do I fix this?

nerdinary
  • 67
  • 2
  • 5
  • who enters `y` in this case? See [Python: read streaming input from subprocess.communicate()](http://stackoverflow.com/a/17698359/4279). Read the links provided there: using `pty` should fix the output written outside of stdout/stderr and the bufferring issue. – jfs Feb 17 '15 at 03:04

1 Answers1

2

This is basically because of the buffering that is there normally at whatever your cmd program has. You have to disable the default buffering happening at that program in order to attain what you are looking for.

In case it is a python file you are running through the cmd like the following, you can add "-u" parameter with the python command to get your desired result

cmd = ["python", "-u", "/link/to/python/file.py"]
myrun(cmd)

As per the man docs of python, 

-u     : unbuffered binary stdout and stderr; also PYTHONUNBUFFERED=x

In case of any file that is to run in the shell, I would say stdbuf command is your safe bet. Set the -o option to 0(zero) along with the command and you are good to go.

cmd = ["sh", "\path\to\sh\file.sh"]
unbuffered_cmd = ["stdbuf", "-o0", "-e0"] + cmd
myrun(unbuffered_cmd)

Please note that stdbuf comes with GNU coreutils. So it may not be platform agnostic.

To be on the safer side even with stdbuf present and "-u" set for python, set the bufsize parameter of Popen to 1

Popen(cmd, stdout=PIPE, stderr=STDOUT, bufsize=1)

I am out of options, if stdbuf is also not helping :-)

thiruvenkadam
  • 4,170
  • 4
  • 27
  • 26
  • Thanks! stdbuf appears to do the trick in ordering! But now I've run into an issue where it waits for the user prompt before displaying the prompt message. – nerdinary Feb 17 '15 at 02:14
  • 1
    @nerdinary: if the child process writes the prompt directly to the terminal then trying to affect its stdout buffer (using `-u` or `stdbuf`) won't help. See [`pty`, `pexpect` solutions](http://stackoverflow.com/a/20509641/4279) instead. – jfs Feb 17 '15 at 03:05