Unable to get output of shell script run from python

Question

I want to run a script multiple times, with different paths as arguments, and see the output.

If I run the script path/lizard with argument path_to_code/code1.cpp in a command prompt

path/lizard path_to_code/code1.cpp

I get output - and I would like to run this script on multiple files.

Looking at this and similar questions, I tried

import os, glob

def run_command(command):
    os.system(command)    

program = '"C:/Python27/Scripts/lizard.bat "'
path = '"path_to_code/*.cpp"'
for path1 in glob.glob(path):
    command = program + path1
    run_command(command)

No output.

import glob, subprocess

def run_command(command):
    p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    out, err = p.communicate()
    print out

program = '"C:/Python27/Scripts/lizard.bat "'
path = '"path_to_code/*.cpp"'
for path1 in glob.glob(path):
    command = program + path1
    run_command(command)

No output.

(of course i want to iterate recursively through directory but that is next step).

How can I get the output of the program running from script ? I think logically both versions should get me the output... What am I doing wrong ?

The trailing space in the double quotes around the path to the python script looks wrong to me. Does removing that help? — Etan Reisner, May 15 '15 at 16:05
http://stackoverflow.com/questions/4760215/running-shell-command-from-python-and-capturing-the-output — Doon, May 15 '15 at 16:05
@EtanReisner without it, the path and the argument are in a single string and there is no such program. — Thalia, May 15 '15 at 16:06
@Doon that is exactly the question I quoted and what I tried to get working, see second example — Thalia, May 15 '15 at 16:06
You need a space *after* the quote I agree but inside the quote seems unlikely (unless you can use `"C:/Python27/Scripts/lizard.bat ""path_to_code/*.cpp"` at your command prompt correctly or python is stripping those quotes somewhere). — Etan Reisner, May 15 '15 at 16:07
You can run `C:/Python27/Scripts/lizard.bat path_to_code/*.cpp` from your command prompt correctly manually, yes? — Etan Reisner, May 15 '15 at 16:12
@Doon I don't see how to write the `subprocess.check_output`... I tried the `out, err = p.communicate()` with no output result — Thalia, May 15 '15 at 16:16
@EtanReisner you were right, about the quotes, I wanted to be sure it is right and overdid it. — Thalia, May 15 '15 at 16:23
@EtanReisner can you please write an answer so I can accept it — Thalia, May 15 '15 at 16:52
Don't update the question with an answer. You can answer your own question with the corrected code and the explanation. — Etan Reisner, May 15 '15 at 16:57

score 1 · Answer 1 · answered May 15 '15 at 16:44

Try using subprocess.check_output

it should do what you want.

[~] cat foo.sh
#!/bin/sh

echo "Hello World!"

[~] python2.7
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> foo = subprocess.check_output('./foo.sh')
>>> foo
'Hello World!\n'
>>>

so in your specific example

def run_command(command):
    return subprocess.check_output(command,stderr=subprocess.STDOUT)

Thank you - My mistake was in how I wrote the arguments, which was caught by Ethan Reisner. This is good learning for me still — Thalia, May 15 '15 at 16:52

score 1 · Answer 2 · answered May 15 '15 at 16:45

1

From what I see you forgot to call communicate on the process. Try

def run_command(command):
    p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    stdout, stderr = p.communicate()
    return iter(stdout, b'')

See https://docs.python.org/2/library/subprocess.html?highlight=popen.communicate#subprocess.Popen.communicate

Cheers

answered May 15 '15 at 16:45

Goldfishslayer

430
4
10

Doon's answer might be more suited for your needs but restricts you to Python 2.7 (if that is any concern to you that is ;)) – Goldfishslayer May 15 '15 at 16:48
Thanks, I did try the communicate... I wrote that in comment... My mistake was in how I wrote the arguments, which was caught by Ethan Reisner – Thalia May 15 '15 at 16:51
Thank you for the example still, I upvoted it since it does give good info. – Thalia May 15 '15 at 16:58

score 0 · Accepted Answer · answered May 15 '15 at 17:02

based on the comments to my question, I tried

program = 'C:/Python27/Scripts/lizard.bat'
...
command = [program, path1]

worked - and then I realized that the quotes were the problem, Etan Reisner was correct. Eliminating them made it work.

Complete corrected code:

import os, subprocess, fnmatch

def run_command(command):
    p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    out, err = p.communicate()
    print out

program = 'C:/Python27/Scripts/lizard.bat'
path = 'path_to_code'
matches = []
for root, dirs, filenames in os.walk(path):
    for filename in fnmatch.filter(filenames, '*.cpp'):
         matches.append(os.path.join(root, filename))

for path1 in matches:
    command = [program, path1]
    run_command(command)

Unable to get output of shell script run from python

3 Answers3