1

test.py file

#test.py

#!/usr/bin/env python3

while True:
    inp = input("input: ")
    print("output: " + inp)

subp.py file:

#subp.py

#!/usr/bin/env python3
from subprocess import Popen, PIPE

cmd = Popen(["python3", "test.py"], stdin=PIPE, stdout=PIPE)

while True:
    print("Writing to stdin")
    cmd.stdin.write("My Input To PIPE")
    print("Done Writing to stdin")
    print("Reading from stdout")
    s = cmd.stdout.readline()
    print("Done reading from stdout") # Never executes this line

The output is the following:

Writing to stdin

Done Writing to stdin

Reading from stdout

I understand that the line s = cmd.stdout.readline() is going to block until EOF is found in the stdout file object.

And if I am right, EOF will never be found in stdout unless stdout gets closed ? Can somebody correct me on this?

Going by my understanding, if I modify test.py

import sys

while True:
    inp = input("input: ")
    print("output: " + inp)
    sys.stdout.close() # Added this line hoping to unblock stdout.readline()

Nothing changes, cmd.stdout.readline() still is looking for EOF even though the stdout file is closed ?

What am I missing? I am more concerned about the theory rather done just making it work without understanding. Thank you for the explanations

Also if modify subp.py and add the line cmd.stdout.close() before the line s = cmd.stdout.readline(), it throws an error saying that I tried reading from a closed file object, which makes sense, but how come it did not throw an error when I close the stdout file object in the test.py file by adding the line sys.stdout.close(). Are these two stdout different things?

Omnistriker
  • 71
  • 6
  • 2
    You need `cmd.stdin.close()` after you are done writing. Note, however, that writing to the cmd's stdin without simultaneously reading from its stdout (and stderr if that is piped, though it's not here) risks deadlock. For this reason, the subprocess routines can do the stdin-writing for you. – torek Dec 27 '18 at 22:15
  • Can you explain the theory why deadlock would occur? If I write to stdin, then `test.py` will see that and simply output to stdout? So I can do `cmd.stdout.read()`. Why would there be a possibility of deadlock? Also why do i have to do `cmd.stdin.close()` ? – Omnistriker Dec 27 '18 at 22:20
  • 1
    Also if your files actually look like that, your `#!` lines aren't doing anything. Those only do anything at the start of a file. It looks like you're actually running your `subp.py` on Python 2, judging by the str/bytes handling. – user2357112 Dec 27 '18 at 22:24
  • The `cmd.stdin.close()` call closes the pipe feeding out from the main process to the subprocess's stdin, so that the subprocess sees EOF. You already said that you need that; that's what this does. (What you are/were missing is that the *child*'s stdout is not the *parent*'s stdout.) As to the deadlock, however, see [my answer](https://stackoverflow.com/a/18423003/1256452) to https://stackoverflow.com/q/18421757/1256452. This is a bit outdated now (python3 has a fancier way to do it) but the core concepts still apply. – torek Dec 27 '18 at 22:31
  • Hi, I read your answer and it gave me a lot of insights and understanding; I also read about how pipes work & i feel very solid about the theory. So now the, how can I change my `subp.py` file so that I can get real time updates from the `while True` loop? Again, the `cmd.stdout.readline()` will get stuck looking for an EOF which it will never find because `test.py` is an infinite loop & the program will never end to close the stdout ( giving EOF ). I can't use `output, err = cmd.communicate(input='blah')` either because it only supplies me the first output. – Omnistriker Dec 28 '18 at 20:02

0 Answers0