How to save python Errors (stderr) to a log file ONLY in case errors actually happen?

Question

I have a python script that runs once a minute. For debugging, I want to create text files with errors/exceptions in case they occur. In case errors are absent, I don't want any log files.

EDIT: I can't wrap all my code in a try/except loop, so this doesn't help.

First, I am saving stderr to .txt doing:

sys.stderr = open('ERRORS_%s.txt' % (infile), 'a')
with open("%s_Error_Messages.txt" % (MSKSN), "a") as myfile:
    myfile.close()

Second, I then try to delete empty "%s_Error_Messages.txt" % (MSKSN) files before any raise SystemExit positions since no error was raised (otherwise, the raise SystemExit position would not have been reached).

if os.path.getsize('ERRORS_%s.txt' % (infile)) == 0:
    os.remove('ERRORS_%s.txt' % (infile))

What I get is WindowsError: [Error 32] The process cannot access the file because it is being used by another process. I assume that this is because stderr is always in use by python.exe as long as the script runs.

For the first step, I also tried:

if sys.stderr != 0: 
   sys.stderr = open('ERRORS_%s.txt' % (infile), 'a')

Which does not work since sys.stderr is never 0.

How to write sys.stderr to .txt only in case of actual errors?

Answer 1

Since the accepted answer simply doesn't actually work, Here is the solution:

You said you don't want try-except mechanism, so you can use sys.excepthook which is called when an "unhandled" exception occurs and prints out a given traceback and exception to sys.stderr. Whenever you run your script, if an exception raises your program terminates(because you didn't catch it) but you can log the error in your file.

import sys

def myexcepthook(e_type, e_value, e_tb, org_excepthook=sys.excepthook):
    with open('errors.txt', 'a') as error_file:
        sys.stderr = error_file
        sys.stderr.write(f'--------------- Exception --------------\n')

        # Now this logs into our 'sys.stderr' which is 'error_file'.
        org_excepthook(e_type, e_value, e_tb)
        sys.stderr.write('\n')


sys.excepthook = myexcepthook

# This terminates the program here
1 / 0

print("You can see this only if an exception doesn't occur")

Note : I could directly write into the file, but I chose redirection to let original sys.excepthook log's it's pretty traceback. If you feel comfortable with traceback module you could handle it that way.

This is the output after running three times:

--------------- Exception --------------
Traceback (most recent call last):
  File "....", line 17, in <module>
    1 / 0
ZeroDivisionError: division by zero

--------------- Exception --------------
Traceback (most recent call last):
  File "....", line 17, in <module>
    1 / 0
ZeroDivisionError: division by zero

--------------- Exception --------------
Traceback (most recent call last):
  File "....", line 17, in <module>
    1 / 0
ZeroDivisionError: division by zero

Answer 2

Unless you have a huge volume of error data to worry about, you can just redirect stderr into a stringIO object and then write it out (or not) depending on contents

import StringIO
import sys

sys.stderr = StringIO.StringIO()
# do stuff
if sys.stderr.getvalue():
   with open ("path/to/log.txt") as output:
        output.write(sys.stderr.getvalue())

This is a great case for a context manager that will set the replacement stderr, run your code, then reset stderr back to sys.__stderr__ when done.

If you can't wrap the code in a try, you could also try setting this functionality using sys.excepthook