65

Yesterday, I wrote and ran a python script which executes a shell using subprocess.Popen(command.split()) where command is string which constitutes .sh script and its argument. This script was working fine until yesterday. Today, I ran the same script and now I am continuously hitting this error.

p=subprocess.Popen(shell_command.split())
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 8] Exec format error

I know there are similar questions that have been asked before related to this question, but in my case I tried everything which doesn't solve my purpose. Using shell=True does not work because my shell script calls an another shell script before which some environment has to be set in order to run that script. I am badly stuck in this. I just restart my system once. I am using ubuntu 12.04

EDIT:

 import subprocess
 import os
 import sys

 arg1=sys.argv[1]
 arg2=sys.argve[2]

 shell_command = 'my_path/my_shell.sh ' + arg1 + ' '+ arg2
 P = subprocess.Popen(shell_command.split())
 P.wait()

my_shell.sh:

  arg1=$1
  arg2=$2

  cd $TOP
  setup the environment and run shell script
  build the kernel ...
  execute shell command .....
Saca
  • 10,355
  • 1
  • 34
  • 47
Amit Sharma
  • 1,987
  • 2
  • 18
  • 29
  • Have you tried `import shlex` and using `shlex.split(shell_command)`? Docs: https://docs.python.org/2/library/shlex.html – Curtis Mattoon Nov 07 '14 at 18:52
  • Yes, it gives same error. :( – Amit Sharma Nov 07 '14 at 18:56
  • 1
    Can you post your shell_command and the rest of the (relevant) code? Also, the shebang line of your script? – Curtis Mattoon Nov 07 '14 at 19:00
  • @Curtis Matttoon ping.. – Amit Sharma Nov 07 '14 at 19:19
  • 1
    What's the first line of your script? (#!/usr/bin/...?) Does the shell script actually exist in the path indicated? You might be better served with using absolute paths instead of 'my_path/my_shell.sh'. Ensure you have shebangs in both the python and shell script and they both have the execute bit set. If that still fails, you might have some strange whitespace or something before the 1st line. – Curtis Mattoon Nov 07 '14 at 19:24
  • yes, it has `#!/usr/bin/sh` in shell script and `#/usr/bin/python`in python. But point is this very same script was running before rebooting. And both file are set: chmod 777 file – Amit Sharma Nov 07 '14 at 19:27
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/64509/discussion-between-curtis-mattoon-and-amit-sharma). – Curtis Mattoon Nov 07 '14 at 19:35
  • Thanks @CurtisMattoon I did re-installation and this error is gone. :) – Amit Sharma Nov 08 '14 at 06:39

8 Answers8

130

I solved this by putting this line at the top of the called shell script:

#!/bin/sh

That will guarantee that the system always uses the correct interpreter when running your script.

William Pietri
  • 3,573
  • 5
  • 26
  • 25
  • 3
    Worked! Thank you! – dwn Dec 09 '18 at 14:52
  • 2
    Got a similar problem ([Errno 8] Exec format error) which was caused by the lack of a proper shebang while the execute bit on the file was set. After `chmod -x` the problem was solved. This answer put me on the right track, so thanks :) – eigil Jun 03 '19 at 13:45
  • Instead you can also call your script this way `subprocess.call(["sh", ./test.sh])` to specify which interpreter to use from within python. Explicit over implicit. – anilbey Jul 29 '20 at 09:35
24

Following statement worked for me

subprocess.Popen(['sh','my_script.sh'])

neau
  • 317
  • 1
  • 12
  • The only caveat with this solution is that 'sh' should be in the path, which is not the case usually. – nyuwec Apr 01 '16 at 12:42
  • this was preferable for me as I was working with a shell script I didn't want to alter. – Harry Moreno Apr 24 '17 at 18:46
  • better to use full path to the shell and the script especially if the script is going to be executed from crontab thereby insulating yourself from changes in your $PATH environment variable, to wit: subprocess.Popen(['/bin/bash','//my_script.sh']) – mcchiz Jun 03 '17 at 17:31
  • 2
    @nyuvec Not sure which platform you're on; anything remotely Unix-y will definitely have `sh` in the `PATH`. – tripleee Aug 13 '17 at 09:13
4

The error message suggests that the external program is not a valid executable.

tripleee
  • 175,061
  • 34
  • 275
  • 318
4

As @tripleee said, there is an issue executing your script. Make sure:

  • Change the shell command to "./my_path/my_script.sh" or "/bin/bash my_path/my_script.sh". Account for environment variables, if necessary.
  • Both scripts have execute bit set (chmod +x)
  • The files exist at the location you think they do. (Use abspath or verify environment)
  • The files have contents
  • Try removing and re-typing the first line. I recommend killing the whole line, and hitting backspace several times in case there's a non-printable character before the #!

See also: Why is '#!/usr/bin/env python' supposedly more correct than just '#!/usr/bin/python'?

Community
  • 1
  • 1
Curtis Mattoon
  • 4,642
  • 2
  • 27
  • 34
4

This can also happen if the binary is not meant to run on your system.

I'm on OSX, but the binary I was running is not meant for OSX, as I saw from using file path/to/binary:

webui/bin/wkhtmltopdf: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.18, BuildID[sha1]=b6566a9e44c43a0eebf18d8c1dc6cb616801a77e, stripped
Joe Flack
  • 866
  • 8
  • 14
3

The error is because the executables are not given in the prescribed format for subprocess to execute it.

example:

shell_command1 = r"/path/to/executable/shell/file"

shell_command2 = r"./path/to/executable/shell/file"

subprocess.call(shell_command1) or subprocess.Popen(shell_command1) will not be able to run shell_command1 because subprocess needs an executable command like (mailx, ls, ping, etc.) or executable script like shell_command2 or you need to specify command like this 

subprocess.call(["sh", shell_command1])
subprocess.Popen(["sh", shell_command1])

but however, you can use os.system() or os.Popen() as well

jalazbe
  • 1,801
  • 3
  • 19
  • 40
Dhinesh Sunder Ganapathi
  • 131
  • 8
0

I am currently facing the same issue. I noticed that using shell=True argument, like subprocess.Popen(shell_command.split(), shell=True) works as inteded.

gautam-404
  • 11
  • 4
-1

It is recommended to install the package binfmt-support to help the system better recognize the scipts. It helps regardless of whether they have a shebang line.

Stanislav Fyodorov
  • 83
  • 1
  • 6