How can os.popen be set to use Bash?

Question

I have the following function that is used to execute system commands in Python:

def engage_command(
    command = None
    ):
    #os.system(command)
    return os.popen(command).read()

I am using the os module instead of the subprocess module because I am dealing with a single environment in which I am interacting with many environment variables etc.

How can I use Bash with this type of function instead of the default sh shell?

Answer 1

output = subprocess.check_output(command, shell=True, executable='/bin/bash')

os.popen() is implemented in terms of subprocess module.

---

I am dealing with a single environment in which I am interacting with many environment variables etc.

1. each os.popen(cmd) call creates a new /bin/sh process, to run cmd shell command.

   Perhaps, it is not obvious from the os.popen() documentation that says:

   Open a pipe to or from command cmd

   "open a pipe" does not communicate clearly: "start a new shell process with a redirected standard input or output" -- your could report a documentation issue.

   If there is any doubt; the source confirms that each successful os.popen() call creates a new child process

2. the child can't modify its parent process environment (normally).

Consider:

import os
#XXX BROKEN: it won't work as you expect
print(os.popen("export VAR=value; echo ==$VAR==").read())
print(os.popen("echo ==$VAR==").read())

Output:

==value==

====

==== means that $VAR is empty in the second command because the second command runs in a different /bin/sh process from the first one.

To run several bash commands inside a single process, put them in a script or pass as a string:

output = check_output("\n".join(commands), shell=True, executable='/bin/bash')

Example:

#!/usr/bin/env python
from subprocess import check_output

output = check_output("""
    export VAR=value; echo ==$VAR==
    echo ==$VAR==
    """, shell=True, executable='/bin/bash')
print(output.decode())

Output:

==value==
==value==

Note: $VAR is not empty here.

If you need to generate new commands dynamically (based on the output from the previous commands); it creates several issues and some of the issues could be fixed using pexpect module: code example.