Code being stuck in the loop while reading from process output

Question

This code never prints "hello". Not sure why?

proc = subprocess.Popen(
'./lite-client -C ton-lite-client-test1.config.json -D ./ton-db-dir',
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
shell=True)

line = proc.stderr.readline()
while line != "":
    print(line)
    print(len(line))
    line = proc.stderr.readline()
print("hello")

Once it prints everything from stderr it's still not printing "hello". stderr is finite, it prints about 20 lines on startup.

Any thoughts?

Update: stderr comes from long running process. I want to interact with this process, send commands and read output.

Update 2: I made workaround that works, I added break if string contains specific substring.

*"stderr comes from long running process"* But that doesn't tell us what the stderr was, and it isn't reproducible. On SO you're required to post a repducible example [MCVE](https://stackoverflow.com/help/minimal-reproducible-example) — smci, Jan 04 '20 at 19:47
I was able to easily repro with `perl -le 'for my $i (1..20) { warn "Not hello $i"; sleep 1 }'` — tripleee, Jan 04 '20 at 19:49

tripleee · Accepted Answer · 2020-01-05T14:53:11.050

I was able to repro the problem by substituting

subprocess.Popen(
    ['perl', '-e', 'for my $i (1..20) { warn "Not hello $i"; sleep 1 }'],
    ...) # no shell=True; see below

The output from stderr is bytes here. You can add text=True to the keyword arguments, or change to while line != b"".

Also, tangentially, you can easily get rid of the pesky shell=True.

import subprocess

proc = subprocess.Popen(
  ['./lite-client', '-C', 'ton-lite-client-test1.config.json', '-D', './ton-db-dir'],
  stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
  text=True)

line = proc.stderr.readline()
while line != "":
    print(line, end='')  # line already has a newline
    print(len(line))
    line = proc.stderr.readline()
print("hello")

I would perhaps also change to while True: and then break when the line is empty, so that you don't have any repeated code. I didn't make that change here just to make it easier to compare our programs.

Suggest you should add the repro instructions to this answer. — smci, Jan 05 '20 at 09:31

Code being stuck in the loop while reading from process output

1 Answers1