Python: capture and save function output

Question

I want to capture the output of a function with multiple debug print functions, then save the output to a file. I need a pythonic way to achieve this (I cannot use python script.py > out.txt)

For instance, I might have a piece of code of the form:

import SomeClass

my_class = SomeClass()
for i in range(0, 9999):
    result = my_class.function_to_test(some_variable=i)
    *do something else with the result*

where I want to capture the printed debug output statements from function_to_test. (Note: In reality, the printed outputs are not the result of a simple for loop changing a variable).

From How to capture stdout output from a Python function call?, I have tried to use:

import SomeClass

class Capturing(list):
    def __enter__(self):
        self._stdout = sys.stdout
        sys.stdout = self._stringio = StringIO()
        return self
    def __exit__(self, *args):
        self.extend(self._stringio.getvalue().splitlines())
        del self._stringio    # free up some memory
        sys.stdout = self._stdout

my_class = SomeClass()
for i in range(0, 9999):
    with Capturing() as output:
        result = my_class.function_to_test(some_variable=i)
    with open(str(i), "w") as f:
        f.write(output)

However, this does not work in my case. Nor do I want every line stored as a list (I think this would disrupt how the printout would look). Again, I cannot simply do python script.py > out.txt. For one reason, the real calculations are run in parallel and outputs are returned in a jumbled mess.

How can I save the output of function_to_test to a file?

if you are doing asynchronous stuff like this, you should look into the `asyncio` library and `async` in general. It outperforms threading and you can make much more custom setups. — coderman1234, Mar 30 '21 at 14:51

Vincent Bénet · Accepted Answer · 2021-04-08T14:00:09.457

First create an objet that will store your outputs with a write method:

class EmittingStream:
    def __init__(self, path_logs):
        self.path_logs = path_logs

    def write(self, text):
        f = open(self.path_logs, "a")
        f.write(text)
        f.close()

Then associate this class with stdout:

sys.stdout = EmittingStream("my/path/to/logs.txt")

Here a full mini example with a decorator:

def logger(f):
    class EmittingStream:
        def __init__(self, path_logs):
            self.path_logs = path_logs

        def write(self, text):
            f = open(self.path_logs, "a")
            f.write(text)
            f.close()

    def wrapper(*args, **kwargs):
        save_std_out = sys.stdout
        sys.stdout = EmittingStream(os.path.join(dir, "test.txt"))
        res = f()
        sys.stdout = save_std_out
        return res
    return wrapper

@logger
def f():
    print("ok")
 
if __name__ == "__main__":
    f()

Not that you should save the default std_out to restore it after.

Python: capture and save function output

1 Answers1