Running commands/programs with subprocess

Question

I am new to the subprocess module in python.

The documentation provided this example:

>>> subprocess.check_output(["echo", "Hello World!"])
b'Hello World!\n'

What I tried is:

>>> import subprocess
>>> subprocess.check_output(["cd", "../tests", "ls"])
/usr/bin/cd: line 4: cd: ../tests: No such file or directory
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['cd', '../tests', 'ls']' returned non-zero exit status 1

I am confused because this is my file structure:

   /proj
      /cron
         test_scheduler.py
      /tests
         printy.py
         test1.py
         test2.py
         ...

These are my other attempts as well:

>>> subprocess.check_output(["cd", "../tests", "python", "printy.py"])
/usr/bin/cd: line 4: cd: ../tests: No such file or directory
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['cd', '../tests', 'python', 'printy.py']' returned non-zero exit status 1


>>> subprocess.check_output(["cd", "../tests;", "ls"])
/usr/bin/cd: line 4: cd: ../tests;: No such file or directory
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['cd', '../tests;', 'ls']' returned non-zero exit status 1

score 3 · Accepted Answer · edited May 23 '17 at 10:32

3

Try to avoid shell=True if possible.

In this case, you can certainly avoid. The problem you are facing is: cd is a shell builtin. Its not a command/program/utility that can be called from outside. You need to be within a shell for cd to work. What you can instead do is change your current directory. Execute the command. And then go back to your original directory.

You'll need to do something like the below:

pathBefore = os.getcwd()
os.chdir("/path/to/your/directory")
subprocess.check_output(["ls"])
os.chdir(pathBefore) # get back to the path we were in before

UPDATE: A better approach pointed out by @JFSebastian is to use the additional cwd argument to check_output call.

edited May 23 '17 at 10:32

Community

1
1

answered Feb 13 '15 at 19:11

UltraInstinct

43,308
12
81
104

That is probably why I adopted to write Shell=True - as one of my first attempts was a cd. Now all pieces of the puzzle fall in place :) – BitTickler Feb 13 '15 at 19:12
If you are not doing anything with the shell, you are best away from using `shell=True`. It might seem like it makes life easy for you, but trust me, it might give you headaches in the long run. – UltraInstinct Feb 13 '15 at 19:15
True. My one time I used python was just when I needed some backup script while in a curious mood of trying some other language. So I have no deeper knowledge on best practices with python. – BitTickler Feb 13 '15 at 19:16
`/usr/bin/cd` (shown in the error message) is not a shell builtin. You should [use `cwd` parameter instead of `os.chdir()`](http://stackoverflow.com/a/28508621/4279). – jfs Feb 13 '15 at 20:59
@J.F.Sebastian Thanks! I agree, `cwd` argument is a better choice. I will edit my answer. BTW, great observation about `/usr/bin/cd`. It actually prompted me to read up a bit, and found out that [`cd` is provided both as a shell builtin as well as a regular program.](http://unix.stackexchange.com/a/38819/20629). Interesting. – UltraInstinct Feb 14 '15 at 12:57

score 1 · Answer 2 · answered Feb 13 '15 at 19:04

The relative path to the tests directory depends on where the script is being run from. I would suggest calling subprocess.check_output(["pwd"]) to check where you are.

Also you can't combine two commands in the same call like in your attempt with ["cd", "../tests", "python", "printy.py"]. You'll need to make two separate calls with ["cd", "../tests"] and ["python", "printy.py"] respectively.

score 1 · Answer 3 · edited May 23 '17 at 12:03

The error message is clear:

/usr/bin/cd: line 4: cd: ../tests: No such file or directory

that is you have successfully started /usr/bin/cd program that failed and printed the error message.

If you wanted to run ls command from ../tests directory instead:

import os
import subprocess

cwd = os.path.join(get_script_dir(), '../tests')
output = subprocess.check_output(['ls'], cwd=cwd)

where get_script_dir().

Note: do not use a relative path for the directory -- your script can be run from a different directory -- relative paths fail in this case.

score 0 · Answer 4 · answered Feb 13 '15 at 19:05

0

You are missing a argument here I think.
Here a snippet from the only python script I ever wrote:

#!/usr/local/bin/python

from subprocess import call
...
call( "rm " + backupFolder + "*.bz2", shell=True )

Please note the shell=True in the end of that call.

answered Feb 13 '15 at 19:05

BitTickler

10,905
5
32
53

This is not recommended: http://stackoverflow.com/questions/3172470/actual-meaning-of-shell-true-in-subprocess – jonnybazookatone Feb 13 '15 at 19:09

Running commands/programs with subprocess

4 Answers4