Python subprocess popen pipe

Question

I am trying to issue a command in python and read the input and parse it back. This is the command i am trying to issue (example when running on the command line)

-bash-3.2$ echo "passthru TST,1234" | ./vsh -a 127.0.0.1 -p 7000
PASSTHRU TST,1234
udp_send_command: timed out waiting for response
-bash-3.2$

I am trying to use python subprocess

            vsh_command = ' "passthru ' + cmd + '" | ./vsh -a ' + ip_address + ' -p ' + port
            print vsh_command
            new_command = ['echo', vsh_command]
            print new_command
            proc = subprocess.Popen(new_command,
                    stdout=subprocess.PIPE,
                    stderr=subprocess.PIPE,
                    stdin=subprocess.PIPE,
                    shell=True)
            print '1'
            out_2, err = proc.communicate()
            print out_2
            print '2'
            rcv = out_2.splitlines()
            print '3'
            for r in rcv:
                    print '     ' + r
            print '4'

`

 "passthru TST,1234" | ./vsh -a 127.0.0.1 -p 7000
['echo', ' "passthru TST,1234" | ./vsh -a 127.0.0.1 -p 7000']
1


2
3

4

Is a | command not possible with subprocess? if so is there another way that i can run this shell command and get the input?

UPDATE: (With single string for shell=true command)

`vsh_command = 'echo "passthru ' + cmd + '" | ./vsh -a ' + ip_address + ' -p ' + port
            print vsh_command
            proc = subprocess.Popen(vsh_command,
                    stdout=subprocess.PIPE,
                    stderr=subprocess.PIPE,
                    stdin=subprocess.PIPE,
                    shell=True)
            print '1'
            out_2, err = proc.communicate()
            print out_2
            print '2'
            rcv = out_2.splitlines()
            print '3'
            for r in rcv:
                    print '     ' + r
            print '4'

`

Run (python script)

echo "passthru TST,1234" | ./vsh -a 127.0.0.1 -p 7000
1
PASSTHRU TST,1234

2
3
     PASSTHRU TST,1234
4
SUCCESS

Still not quite what im looking for, looks like its just doing the echo command not the pipe

EDIT #2

            p = Popen(["./vsh", "-a", ip_address, "-p", str(port)],stdin=PIPE, stdout=PIPE)
            out, err =  p.communicate("passthru TST,1234")
            print '----------'
            print out

Output:

-bash-3.2$ python mfd_ascii_test.py
udp_send_command: timed out waiting for response
----------
PASSTHRU TST,1234

-bash-3.2$

out is the result of the echo, not the 'udp_send_command: timed out waiting for response'

You need to pass a single string with shell=True to pipe or use two processes piping the stdout of the first to the stdin of the second — Padraic Cunningham, Sep 23 '15 at 20:47
*"Still not quite what im looking for"* -- what do you expect to happen instead? — jfs, Sep 23 '15 at 21:14
This is the result i get when i run on the command line: udp_send_command: timed out waiting for response. But it looks like i just get the results of the eqcho — georges, Sep 23 '15 at 21:25
@ Padraic Cunningham vsh is a script to send udp packets that we use — georges, Sep 23 '15 at 22:49
unrelated: to avoid doubling newlines, use `print s,` (note: comma) -- it is [softspace](https://docs.python.org/2.4/lib/bltin-file-objects.html#l2h-256) hack. btw, do you want both to capture stdout/stderr as strings *and* to print them? If all you want is to print them then I don't understand why do you need to capture them, just remove `stdout=PIPE` and the output will appear on the screen. — jfs, Sep 23 '15 at 23:18
I want to parse the output of the script in the python script — georges, Sep 23 '15 at 23:26

score 2 · Answer 1 · answered Sep 23 '15 at 20:52

2

If you want to pipe the output from one to the other, you can pipe the stdout from one process to the stdin of another:

from subprocess import PIPE,Popen

p1 = Popen(["echo", "passthru TST,1234"],stdout=PIPE)

p2 = Popen(["./vsh", "-a", "127.0.0.1", "-p", "7000"],stdin=p1.stdout,stdout=PIPE,stderr=PIPE)
p1.stdout.close()
out, err = p2.communicate()

If you wanted to use the pipe character, you would need to set shell=True and pass a single string:

from subprocess import check_output

out = check_output('echo "passthru TST,1234" | ./vsh -a 127.0.0.1 -p 7000',shell=True)

answered Sep 23 '15 at 20:52

Padraic Cunningham

176,452
29
245
321

-bash-3.2$ echo "passthru TST,1234" | ./vsh -a 127.0.0;1 -p 7000 udp_send_command: timed out waiting for response................... `-bash-3.2$ python mfd_ascii_test.py ---- ,` and the code `p1 = Popen(["echo", "passthru TST,1234"],stdout=PIPE) p2 = Popen(["./vsh", "-a", "127.0.0.1", "-p", "7000"],stdin=p1.stdout,stdout=PIPE,stderr=PIPE)p1.stdout.close() out, err = p2.communicate() print '----' print out` – georges Sep 23 '15 at 22:43
should I do `p2.stdout.close()` after calling `p2.communicate()`? @Padraic Cunningham – alper Jun 13 '20 at 12:03
1

@alper, not needed. As per the docs you `Allow p1 to receive a SIGPIPE if p2 exits.` `stdout` and `stderr` are both closed regardless in `communicate`. – Padraic Cunningham Jun 13 '20 at 17:08

score 1 · Accepted Answer · edited May 23 '17 at 12:00

1

To pass a string as an input to a subprocess and to capture its output, you could use Popen.communicate() method:

#!/usr/bin/env python2
from subprocess import Popen, PIPE, STDOUT

p = Popen(["./vsh", "-a", ip_address, "-p", str(port)],
          stdin=PIPE, stdout=PIPE, stderr=STDOUT)
output = p.communicate("passthru " + cmd)[0]

stderr=STDOUT merges stderr into stdout.

Set stdout=PIPE to capture stdout, set stderr=PIPE to capture stderr; otherwise communicate() returns None for the corresponding pipes. If the process may write outside stdout, stderr then use pexpect or pty + select modules. Here's pexpect.runu() and pty.spawn() code examples.

edited May 23 '17 at 12:00

Community

1
1

answered Sep 23 '15 at 21:12

jfs

399,953
195
994
1,670

running python 2.4, doesn't like str object has no attribute format – georges Sep 23 '15 at 22:48
How do i get the output? `out, err = p.communicate("passthru IDN?,1234") print '----------' print out` `-bash-3.2$ python mfd_ascii_test.py PASSTHRU IDN?,1234 udp_send_command: timed out waiting for response ---------- None ` – georges Sep 23 '15 at 23:02
@georges: include *minimal* code example and show its result in the question (to enable proper formatting). Set `stdout=PIPE` to capture stdout, set `stderr=PIPE` to capture stderr. If the process may write outside stdout, stderr then [use pexpect or pty modules](http://stackoverflow.com/a/12471855/4279) – jfs Sep 23 '15 at 23:03
1

@georges: what about `stderr=PIPE`? – jfs Sep 23 '15 at 23:15

Tibox · Answer 3 · 2015-09-24T00:57:01.283

-1

You can use directly a string command with subprocess.check_output if you set shell=True it's easier this way (python 2.7).

output = subprocess.check_output(" ".join(new_command), shell=True)

Or you can use subprocess.Popen (python < 2.7).

output, err = subprocess.Popen(" ".join(new_command), stdout=subprocess.PIPE, shell=True).communicate()

edited Sep 24 '15 at 00:57

answered Sep 23 '15 at 21:09

Tibox

827
6
7

in this case use `output, err = subprocess.Popen(" ".join(new_command), stdout=subprocess.PIPE, shell=True).communicate()` it'll get the job done. – Tibox Sep 24 '15 at 00:55

Python subprocess popen pipe

3 Answers3