0

What specific syntax must be changed below in order to get the call to subprocess.popen to retry if no response is received in n seconds?

def runShellCommand(cmd):
  process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
  process.wait()

The problem we are having is that the command is succeeding but the command is not receiving a response. This means that the runShellCommand(cmd) function is just hanging forever.

If the process.wait() lasted only n seconds and then retried running the same cmd, then repeated the call/wait cycle 3 or 4 times, then the function could either receive a response from one of the subsequent tries and return successful, or could fail gracefully within a specified maximum period of time.

CodeMed
  • 9,527
  • 70
  • 212
  • 364
  • does [this answer check_output](https://stackoverflow.com/a/12698328/1518100) help? – Lei Yang Feb 17 '22 at 04:33
  • @LeiYang The problem is that it is not returning anything at all. It just hangs forever. I did scan all the answers in the link you posted, but that OP is 12 years old and there are 35 answers, which are mostly outdated workarounds with unnecessary complexity. Another answer there says `subprocess.run` replaced `check_output` in python3. It would be helpful to see a clear, modern answer in python3. – CodeMed Feb 17 '22 at 05:03

1 Answers1

2

Your process is probably deadlocking due to the STDOUT buffer filling up.

Setting stdout=subprocess.PIPE makes the process redirect STDOUT to a file called process.stdout instead of the terminal output. However, process.wait() doesn't ever read from process.stdout. Therefor, when process.stdout fills up (usually after a few megabytes of output), then the process deadlocks. The process is waiting for STDOUT (directed to process.stdout) to get read, but it will never get read because process.wait() is waiting for the process to finish, which it can't do because it's waiting to print to STDOUT... and that's the deadlock.

To solve this and read the output, use something like:

def runShellCommand(cmd):
   return subprocess.run(cmd, shell=True, stdout=subprocess.PIPE, text=True).stdout

Note that text=True requires Python 3.7 or later. Before that, use universal_newlines=True for the same effect, or leave that argument out to get the results as bytes instead.

Security note: Please consider removing shell=True. It's horribly unsafe (subject to the variable expansion whims of your shell, which could be almost anything from a simple POSIX sh or bash, but also something more unusual like tcsh, zsh, or even a totally unexpected custom shell compiled by the user or their sysadmin).

E.g. instead of

subprocess.run('echo "Hello, World!"', shell=True)

You can use this more safely:

subprocess.run(['echo', 'Hello, World!'])
Pi Marillion
  • 4,465
  • 1
  • 19
  • 20
  • How do you suggest modifying your security suggestion to run in any operating system? For example `if platform.system() == 'Windows'` then `subprocess.run()` looks one way and `if platform.system() == 'Linux'` then `subprocess.run()` looks different? – CodeMed Feb 17 '22 at 20:18
  • @CodeMed That's the right way, yes. You can use `sys.platform` or `platform.system()` in an `if sys.platform.startswith('win'): ...` to decide which command to run, or with which arguments to run with. – Pi Marillion Feb 17 '22 at 23:02