Why is my python function being skipped?

Question

I've got a small script that's trying to execute an external command. But for some reason, the function that I made to execute the command is being completely skipped over! No errors seem to be raised, it just doesn't execute. I've got a few debug print statements inside it to verify that the function gets entered, but they never print. And I've got a print statement outside of it to verify that the script isn't dying. So what gives?

from xml.etree import ElementTree as et
import subprocess

pomFileLocation = "pom.xml"
uiAutomationCommand = "mvn clean install"
revertPomFileCommand = "git checkout pom.xml"
profileToSetToDefault = "smoketest"

def modifyxml( datafile, value ):
    print( "modifying " + datafile )
    tree = et.parse( datafile )
    rootNodes = tree.getroot()
    for node in rootNodes:
        if "profiles" in node.tag:
            for profile in node.iter():
                foundIt = False
                for param in profile.iter():
                    if "id" in param.tag and profileToSetToDefault in param.text:
                        foundIt = True
                        break
                if foundIt == True:
                    for param in profile.iter():
                        if "activation" in param.tag:
                            for child in param.iter():
                                if "activeByDefault" in child.tag:
                                    child.text = value
                                    tree.write( datafile )
                                    return

def runExternalCommand( comm ):
    print( "running command " + comm )
    p = subprocess.Popen( comm, bufsize=-1, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE ).communicate()[0]
    print( str(p) )
    while( True ):
        print( "still running" )
        retcode = p.poll()
        line = p.stdout.readline()
        yield line
        if( retcode is not None ):
            print("Exiting")
            break   
    return

if __name__ == '__main__':
    modifyxml( pomFileLocation, "true" )
    #runExternalCommand( uiAutomationCommand )
    runExternalCommand( revertPomFileCommand )
    print( "finished" )

where do you get `runExternalCommand()` definition? It is broken. Please, leave a comment where you get it so that others won't use the broken code. [my answer provides some alternatives](http://stackoverflow.com/a/28352727/4279) — jfs, Feb 05 '15 at 19:49

score 2 · Accepted Answer · answered Feb 05 '15 at 18:46

runExternalCommand uses yield, so if you want it to execute all the way to the end, you ought to call it like for something in runExternalCommand(revertPomFileCommand):. Or just delete the yield line, since you don't seem to need it anyway.

def runExternalCommand( comm ):
    print( "running command " + comm )
    p = subprocess.Popen( comm, bufsize=-1, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE ).communicate()[0]
    print( str(p) )
    while( True ):
        print( "still running" )
        retcode = p.poll()
        line = p.stdout.readline()
        yield line
        if( retcode is not None ):
            print("Exiting")
            break   
    return

if __name__ == '__main__':
    modifyxml( pomFileLocation, "true" )
    #runExternalCommand( uiAutomationCommand )
    for line in runExternalCommand( revertPomFileCommand ):
        pass
    print( "finished" )

Or

def runExternalCommand( comm ):
    print( "running command " + comm )
    p = subprocess.Popen( comm, bufsize=-1, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE ).communicate()[0]
    print( str(p) )
    while( True ):
        print( "still running" )
        retcode = p.poll()
        line = p.stdout.readline()
        if( retcode is not None ):
            print("Exiting")
            break   
    return

if __name__ == '__main__':
    modifyxml( pomFileLocation, "true" )
    #runExternalCommand( uiAutomationCommand )
    runExternalCommand( revertPomFileCommand )
    print( "finished" )

Ah. That's what I get for blindly copying and pasting from the internet. I didn't know yield would do that. I removed it, and now it works. Thanks! — user2623609, Feb 05 '15 at 18:50
To clarify, it's not that the function runs up to the yield-statement. Rather, it won't run at all until it is iterated over. The bits before the yield-statement also don't get run. — Steven Rumbalski, Feb 05 '15 at 18:50
Please, add ALL CAPSLOCK DISCLAIMER if you knowingly use broken code. — jfs, Feb 05 '15 at 19:54

score 1 · Answer 2 · edited May 23 '17 at 12:29

As @Kevin said, the main (but not the only) issue is that runExternalCommand is a generator. To consume it, you could run: print(list(runExternalCommand(revertPomFileCommand))).

Though the function runExternalCommand() is broken: there is no point to call p.stdout.readline() after .communicate() returns (the latter waits for the child process to finish and returns the whole output at once).

It is not clear what result you want to get e.g., to run the git command and to store its output in a variable, you could use subprocess.check_output():

from subprocess import check_output, STDOUT

output = check_output("git checkout pom.xml".split(),
                      stderr=STDOUT, universal_newlines=True)

To discard child's stdout/stderr instead of saving it, use subprocess.check_call():

from subprocess import check_call, DEVNULL, STDOUT

check_call("git checkout pom.xml".split(),
           stdout=DEVNULL, stderr=STDOUT)

For the code example, to read output while the child process is still running, see Constantly print Subprocess output while process is running.

Why is my python function being skipped?

2 Answers2