0

A couple of answers (first, second) have mentioned that subprocess.Popen is a non blocking call.

What can be a simple example which can validate it or can be used to explain it to a beginner.

I tried the following code. It shows that "Finish" is printed before printing output of ls -lrt but as soon as I add sleep 10 before ls -lrt in command, it waits for command to finish. 

import logging
import os
import subprocess
import signal
import time

log = logging.getLogger(__name__)


class Utils(object):

    @staticmethod
    def run_command(cmnd, env=None, cwd=None, timeout=0):
        p = subprocess.Popen(cmnd, shell=True, stdin=None, bufsize=-1, env=env,
                             stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
                             close_fds=True, cwd=cwd, preexec_fn=os.setsid)

        #stdout_val = p.communicate()[0]
        stdout_val = p.stdout.read()
        return p.returncode, stdout_val.strip()

if __name__ == '__main__':
    print "Start"
    print "Invoke command"
    status, output = Utils.run_command("ls -lrt")   # line - 10
    #status, output = Utils.run_command("sleep 10;ls -lrt") # line - 11
    for i in xrange(10):
        print "Finish"
    print status
    print output

EDIT 1: Replacing call p.communicate() with p.stdout.read() after suggestion.

Community
  • 1
  • 1
ViFI
  • 971
  • 1
  • 11
  • 27
  • 1
    `p.communicate` blocks which means that `Utils.run_command` will block until the command finishes. – mgilson Nov 15 '16 at 22:52
  • @mgilson : Thanks for catching that. But if I replace `p.communicate()` with `p.stdout.read()` even then behavior is same. – ViFI Nov 15 '16 at 23:02
  • 1
    @ViFI, of course it does, because `read()` blocks too (it doesn't return until it's able to get all the way to EOF). – Charles Duffy Nov 15 '16 at 23:09
  • ...by the way, if you're trying to teach things to beginners, please avoid showcasing bad practices such as use of `shell=True`. If you're creating a general-purpose utility function, it's better to do the safe and performant thing by default, and let your callers explicitly invoke a shell if they need one (as by using `["sh", "-c", "sleep 10; ls -lrt"]`). – Charles Duffy Nov 15 '16 at 23:10
  • @CharlesDuffy : and that's where a beginner's book ends :) Is there anything else I can do to make it non blocking ? Thanks. – ViFI Nov 16 '16 at 00:44
  • @ViFI, well, sure, but what do you want to do instead of blocking? Do you want to read only one line (and block if there's not a line of content yet, but otherwise proceed)? Do you want to read the amount of content that's already-been written? Do you want some other random thing I haven't thought to propose? :) – Charles Duffy Nov 16 '16 at 00:49
  • actually, better to just point to a duplicative question, rather than draw out with details here: http://stackoverflow.com/questions/375427/non-blocking-read-on-a-subprocess-pipe-in-python – Charles Duffy Nov 16 '16 at 00:51
  • @CharlesDuffy: Actually this code has to be used by a non blocking process. If this code is blocking it basically nullifies the "non blocking" nature of parent process. – ViFI Nov 16 '16 at 00:52
  • In that case you should **really** be reading the linked question. Whatever other non-blocking code you're using will presumably already be using asyncio, or twisted, or another such option; you should be following its lead, not trying to do your own thing. – Charles Duffy Nov 16 '16 at 01:14
  • ...which is to say: If you're going to be called by other non-blocking code, then you need to play nice with its abstractions. That might mean you're obliged to be a coroutine rather than a regular function, or obliged to return a channel, or one of many other things. Details matter. – Charles Duffy Nov 16 '16 at 01:15
  • (...and moreover, this means you lied to us when you said that the purpose of this was to be a training piece demonstrating that the creation of a Popen object was a non-blocking operation; if you need to actually use it in a non-training role, that's a different question with a larger set of constraints, as referenced above). – Charles Duffy Nov 16 '16 at 01:18

0 Answers0