Capture stdout stderr of python subprocess, when it runs from cron or rc.local

Question

I have problem accessing output (stderr stdout) of a command when I do lunch it via cron or rc.local

It works perfectly form regular shell, but fails via rc.local

cat /root/watchdog.py 
import subprocess
    cmd = ( 'echo "TEST" |gnokii --config /root/.config/gnokii/config --sendsms +123456789xx ')
    #p = subprocess.Popen([cmd, '2>&1'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
    p = subprocess.Popen([cmd, '2>&1'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
    output = p.stdout.read()
    output += p.stderr.read()
    logFile = open("/root/logfile", 'a+')
    ##### 
    #Idea to read line by line:
    #output = ''
    #    for line in iter(p.stdout.readline,''):
    #       print "captured line: %s" % line.rstrip()
    #       #logFile.write(line.rstrip())
    #       output += line
    logFile.write(output)
    logFile.close()

The output when run from console looks like:

/root/watchdog.py 
GNOKII Version 0.6.30
Cannot open logfile /root/.cache/gnokii/gnokii-errors
WARNING: cannot open logfile, logs will be directed to stderr
Send succeeded with reference 186!

in my rc.local

/root/watchdog.py  > /root/mywatchPY.out 2>&1 & 

This looks interesting: Redirect subprocess stderr to stdout but it does not solve the problem.

Any idea howto capture sdterr/stdout of subprocess run without full shell?

Answer 1

There are multiple issues in your code:

• pass the command and its args as a string when shell=True otherwise the args are passed to the shell itself instead of the command
• you should use stderr=subprocess.STDOUT instead of 2>&1 if you meant to apply the latter to the whole pipeline and not just the last command in it
• use p.communicate() instead of p.stdout.read(), p.stderr.read() otherwise the subprocess may stall if any of OS pipe buffers fill up
• if you want to redirect the output to a file then you do not need to save it as a string first

import shlex
from subprocess import Popen, PIPE, STDOUT

with open("/root/logfile", 'ab', 0) as logfile:
    p = Popen(shlex.split('gnokii ... +123456789xx'), 
              stdin=PIPE, stdout=logfile, stderr=STDOUT)
    p.communicate(b'TEST')

Redirect subprocess stderr to stdout doesn't apply because you redirect stdout explicitly.