81

I have a python script that has to launch a shell command for every file in a dir:

import os

files = os.listdir(".")
for f in files:
    os.execlp("myscript", "myscript", f)

This works fine for the first file, but after the "myscript" command has ended, the execution stops and does not come back to the python script.

How can I do this? Do I have to fork() before calling os.execlp()?

kaiete
  • 23
  • 10
Davide Gualano
  • 12,813
  • 9
  • 44
  • 65
  • [`returncode = subprocess.call(['myscript', f])`](http://stackoverflow.com/a/5184921/4279) – jfs Mar 05 '14 at 15:25

7 Answers7

126

subprocess: The subprocess module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes.

http://docs.python.org/library/subprocess.html

Usage:

import subprocess
process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)
process.wait()
print process.returncode
Serge
  • 1,947
  • 3
  • 26
  • 48
user39307
  • 1,596
  • 1
  • 9
  • 6
  • 1
    I think as the accepted answer, this should contain at least the same amount of detail as @Harley. This is more of a personal request, but I think the newer docs present the information better. Could you link to the 2.7 version of the documentation instead? – Ehtesh Choudhury Aug 03 '11 at 20:58
  • 3
    The newer version of the documentation is located at http://docs.python.org/library/subprocess.html . – wanderso Sep 29 '11 at 17:29
  • 4
    remove stdout=PIPE it might hang the script if the child program produces enough output (around 65KB on Linux). `shell=True` is unnecessary. – jfs Nov 22 '12 at 16:11
  • 2
    Without shell=True it raises an exception on python 2.7... File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory – SeF Jun 07 '17 at 14:54
66

You can use subprocess.Popen. There's a few ways to do it:

import subprocess
cmd = ['/run/myscript', '--arg', 'value']
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
for line in p.stdout:
    print line
p.wait()
print p.returncode

Or, if you don't care what the external program actually does:

cmd = ['/run/myscript', '--arg', 'value']
subprocess.Popen(cmd).wait()
Harley Holcombe
  • 175,848
  • 15
  • 70
  • 63
  • 1
    `for line in p.stdout:` might delay the output of a line due to read-ahead. You could use `for line in iter(p.stdout.readline, b''): print line,` (note: comma at the end). Remove stdout=PIPE in the second example; it might hang the script. Or just use `subprocess.call()` if you don't need an exception for non-zero returned code – jfs Nov 22 '12 at 16:17
  • @HarleyHolcombe i tried your solution via `subprocess.Popen("/usr/bin/sleep 20").wait()`, which didn't work (`FileNotFoundError:`), to make it work I had to separate the command into substrings and put them into a list `subprocess.Popen(["/usr/bin/sleep", "20"]).wait()` why is that necessary? – Alexander Cska Jul 23 '22 at 10:41
13

The subprocess module has come along way since 2008. In particular check_call and check_output make simple subprocess stuff even easier. The check_* family of functions are nice it that they raise an exception if something goes wrong.

import os
import subprocess

files = os.listdir('.')
for f in files:
   subprocess.check_call( [ 'myscript', f ] )

Any output generated by myscript will display as though your process produced the output (technically myscript and your python script share the same stdout). There are a couple of ways to avoid this.

  • check_call( [ 'myscript', f ], stdout=subprocess.PIPE )
    The stdout will be supressed (beware if myscript produces more that 4k of output). stderr will still be shown unless you add the option stderr=subprocess.PIPE.
  • check_output( [ 'myscript', f ] )
    check_output returns the stdout as a string so it isnt shown. stderr is still shown unless you add the option stderr=subprocess.STDOUT.
deft_code
  • 57,255
  • 29
  • 141
  • 224
  • 4
    @Luke Stanley: your question is unclear, but I'll give it a shot. The `subprocess.check_*` functions makes calling external binaries easier than manually using `subprocess.Popen`. In particular you never need to deal with `wait()`, `communicate()`, `return_code`, etc. You also don't need to worry about gotchas like the dangling pipe stalling the executable. When it comes to _interacting_ with an external binary the `check_*` functions aren't tremendously helpful, but for most use cases the result is shorter and less error prone, aka easier. – deft_code Jun 10 '11 at 19:03
  • 1
    you could use `subprocess.DEVNULL` or `open(os.devnull, 'r+b')` to ignore input/output instead of subprocess.PIPE to avoid possible dead-lock – jfs Nov 22 '12 at 16:21
7

The os.exec*() functions replace the current programm with the new one. When this programm ends so does your process. You probably want os.system().

  • 1
    os.system() is very problematic in terms of security, to the point where many C libraries just remove this standard C function outright. subprocess.Popen() with shell=False should be used instead – Shnatsel Dec 27 '14 at 00:28
3

use spawn

import os
os.spawnlp(os.P_WAIT, 'cp', 'cp', 'index.html', '/dev/null')
M. Utku ALTINKAYA
  • 2,254
  • 23
  • 29
2

I use os.system

import os
os.system("pdftoppm -png {} {}".format(path2pdf, os.path.join(tmpdirname, "temp")))
Nikolay Frick
  • 2,145
  • 22
  • 17
1

this worked for me fine!

shell_command = "ls -l" subprocess.call(shell_command.split())

pka32
  • 5,176
  • 1
  • 17
  • 21