Interacting with bash from python

Question

I've been playing around with Python's subprocess module and I wanted to do an "interactive session" with bash from python. I want to be able to read bash output/write commands from Python just like I do on a terminal emulator. I guess a code example explains it better:

>>> proc = subprocess.Popen(['/bin/bash'])
>>> proc.communicate()
('user@machine:~/','')
>>> proc.communicate('ls\n')
('file1 file2 file3','')

(obviously, it doesn't work that way.) Is something like this possible, and how?

Thanks a lot

Answer 1

Try with this example:

import subprocess

proc = subprocess.Popen(['/bin/bash'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
stdout = proc.communicate('ls -lash')

print stdout

You have to read more about stdin, stdout and stderr. This looks like good lecture: http://www.doughellmann.com/PyMOTW/subprocess/

EDIT:

Another example:

>>> process = subprocess.Popen(['/bin/bash'], shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
>>> process.stdin.write('echo it works!\n')
>>> process.stdout.readline()
'it works!\n'
>>> process.stdin.write('date\n')
>>> process.stdout.readline()
'wto, 13 mar 2012, 17:25:35 CET\n'
>>> 

Answer 2

Use this example in my other answer: https://stackoverflow.com/a/43012138/3555925

You can get more details in that answer.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import sys
import select
import termios
import tty
import pty
from subprocess import Popen

command = 'bash'
# command = 'docker run -it --rm centos /bin/bash'.split()

# save original tty setting then set it to raw mode
old_tty = termios.tcgetattr(sys.stdin)
tty.setraw(sys.stdin.fileno())

# open pseudo-terminal to interact with subprocess
master_fd, slave_fd = pty.openpty()

# use os.setsid() make it run in a new process group, or bash job control will not be enabled
p = Popen(command,
          preexec_fn=os.setsid,
          stdin=slave_fd,
          stdout=slave_fd,
          stderr=slave_fd,
          universal_newlines=True)

while p.poll() is None:
    r, w, e = select.select([sys.stdin, master_fd], [], [])
    if sys.stdin in r:
        d = os.read(sys.stdin.fileno(), 10240)
        os.write(master_fd, d)
    elif master_fd in r:
        o = os.read(master_fd, 10240)
        if o:
            os.write(sys.stdout.fileno(), o)

# restore tty settings back
termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old_tty)

Answer 3

This should be what you want

    import subprocess
    import threading
    
    p = subprocess.Popen(["bash"], stderr=subprocess.PIPE,shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
    exit = False
    
    def read_stdout():
        while not exit:
            msg = p.stdout.readline()
            print("stdout: ", msg.decode())
    def read_stderro():
        while not exit:
            msg = p.stderr.readline()
            print("stderr: ", msg.decode())
    
    threading.Thread(target=read_stdout).start()
    threading.Thread(target=read_stderro).start()
    
    while not exit:
        res = input(">")
        p.stdin.write((res + '\n').encode())
        p.stdin.flush()

Test result:

>ls
>stdout:  1.py

stdout:  2.py

ssss
>stderr:  bash: line 2: ssss: command not found

Answer 4

I wrote a module to facilitate the interaction between *nix shell and python.

def execute(cmd):
if not _DEBUG_MODE:
    ## Use bash; the default is sh
    print 'Output of command ' + cmd + ' :'
    subprocess.call(cmd, shell=True, executable='/bin/bash')
    print ''
else:
    print 'The command is ' + cmd
    print ''

Check out the whole stuff at github: https://github.com/jerryzhujian9/ez.py/blob/master/ez/easyshell.py

Answer 5

An interactive bash process expects to be interacting with a tty. To create a pseudo-terminal, use os.openpty(). This will return a slave_fd file descriptor that you can use to open files for stdin, stdout, and stderr. You can then write to and read from master_fd to interact with your process. Note that if you're doing even mildly complex interaction, you'll also want to use the select module to make sure that you don't deadlock.