How to collect all the print content and save them in a txt in Python?

Question

I want to collect all of the print content and save them in a txt file. I use this answer's method to generate and collect logs. Also I use a threading, because in my real environment, get_logs and make_logs are in different thread.

import sys
import io
import time
import threading
import traceback

class MyThread(threading.Thread):
    def __init__(self, func, args):
        super(MyThread, self).__init__()
        self.func = func
        self.args = args

    def run(self):
        try:
            self.result = self.func(*self.args)
        except Exception as e:
            f = traceback.format_exc()
            print(f'{f}')
            
    def get_result(self):
        try:
            return self.result
        except Exception:
            return None

def make_logs(delay):
    for i in range(100):
        print(i)
        print("\n")
        time.sleep(delay)


def get_logs(t1):
    if t1.is_alive():
        sys.stdout = old_stdout
        whatWasPrinted = buffer.getvalue()
        with open("output.txt", "w") as text_file:
            text_file.write(whatWasPrinted)
        time.sleep(1)
        get_logs(t1)
    

def do_it():
    t1 = MyThread(make_logs, args=(1,))
    t2 = MyThread(get_logs, args=(t1,))
    t1.start(), t2.start()
    t1.join(), t2.join()


old_stdout = sys.stdout
sys.stdout = buffer = io.StringIO()
do_it()

However, as I execute this code, I can only write the first elment(0) to txt file. Anyone knows why? Thanks.

What do you mean by "print context"? Do you mean the actual output rather than "context"? — tripleee, Jan 12 '22 at 08:45
python comes with a very handy logging module. check it out here https://docs.python.org/3/howto/logging.html — JacoSolari, Jan 12 '22 at 08:50
@tripleee all the print in my code. Should be print content. — haojie, Jan 12 '22 at 09:03
Do you want to redirect all the output to a file or just the output of a specific thread? I do not see the need for two threads here. Moreover, the easiest way would be to use the `file` argument to `print` — azelcer, Jan 18 '22 at 02:21
1)The reason I use thread is because I need it in my real work. 2)I need the output of a specific thread. 3) can you write as an answer? — haojie, Jan 18 '22 at 02:30

Tal Folkman · Answer 1 · 2022-01-12T09:36:49.933

0

you can set sys.stdout to a file object to redirect the outpus:

import sys
sys.stdout = open(PATH_TO_LOG_FILE, "w")
print('test')
do_it()
sys.stdout.close()

you can also use contextlib.redirect_stdout: (works for python 3.4+)

from contextlib import redirect_stdout

with open('help.txt', 'w') as f:
    with redirect_stdout(f):
        print('it now prints to `help.text`')

edited Jan 12 '22 at 09:36

answered Jan 12 '22 at 08:40

Tal Folkman

2,368
1
7
21

Preferably use [`contextlib.redirect_stdout`](https://docs.python.org/3.10/library/contextlib.html#contextlib.redirect_stdout) instead. – AKX Jan 12 '22 at 08:43
Hi, have you tried the second method, use `contextlib.redirect_stdout`, nothing new write in the txt file. – haojie Jan 12 '22 at 09:12
My python version is python3.9 – haojie Jan 12 '22 at 09:20
try use the first method @jasonshu – Tal Folkman Jan 12 '22 at 09:22
@TalFolkman Can you add more details to the first method. A little confused. – haojie Jan 12 '22 at 09:32

AKX · Answer 2 · 2022-01-12T12:11:50.367

You don't need two threads to capture the first N seconds – you can write a filelike object that stops writing to whatever backing object (here it just derives from io.StringIO but could wrap another IO) when it has had enough.

import contextlib
import io
import threading
import time


def make_logs(n, delay):
    for i in range(n):
        print(i)
        print("\n")
        time.sleep(delay)


class StdoutCapturer(io.StringIO):
    def __init__(self, time_limit):
        super().__init__()
        self.time_limit = time_limit
        # This could be initialized later, at the first write:
        self.start_time = time.time()

    def write(self, s: str) -> int:
        elapsed = time.time() - self.start_time
        if elapsed < self.time_limit:
            return super().write(s)
        return 0


if __name__ == "__main__":
    sio = StdoutCapturer(time_limit=3.1)
    with contextlib.redirect_stdout(sio):
        t = threading.Thread(target=make_logs, args=(5, 1))
        t.start()
        t.join()
    print(repr(sio.getvalue()))

This prints out

'0\n\n\n1\n\n\n2\n\n\n3\n\n\n'

EDIT

Based on the comments, I think it wasn't clear what I meant with wrapping another IO, so here's a variant that writes the first N seconds to a file...


class TimeLimitedFileWrapper(io.TextIOWrapper):

    def __init__(self, raw_file, *, time_limit):
        super().__init__(raw_file, encoding="utf-8")
        self.start_time = time.time()
        self.time_limit = time_limit

    def write(self, s: str) -> int:
        elapsed = time.time() - self.start_time
        if elapsed < self.time_limit:
            return super().write(s)
        return 0


if __name__ == '__main__':
    with open("foo.txt", "wb") as f:
        with contextlib.redirect_stdout(TimeLimitedFileWrapper(raw_file=f, time_limit=3.1)):
            t = threading.Thread(target=make_logs, args=(5, 1))
            t.start()
            t.join()

    with open("foo.txt", "rb") as f:
        print(repr(f.read()))

The output is the same.

You could do `with open("something.txt", "w") as f: f.write(sio.getvalue())` at the end – or like my post says, you could instead directly write to the file in your wrapper object. — AKX, Jan 12 '22 at 09:20
Thanks, maybe you ignore what I have write in my question"I may have to collect the first 5 seconds' logs for instance." To be more clear, we need 100 seconds to run `make_logs`, I may have to fetch the first 5 seconds print content form the txt. However, in your method, the txt is generated when `make_logs` finished. — haojie, Jan 12 '22 at 09:30
Yes - this example script runs `make_logs` for 5 seconds, and it logs only the first 3.1 seconds. And _again_, as the prose in my answer says, you could just as well wrap another file object instead of using `StringIO` for buffering. — AKX, Jan 12 '22 at 11:38
Hi, I have tried using a file object to replace StringIO, and it not works. Can you share with me the wat you wrap file object? — haojie, Jan 12 '22 at 14:47
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/241000/discussion-between-jasonshu-and-akx). — haojie, Jan 12 '22 at 14:50

How to collect all the print content and save them in a txt in Python?

2 Answers2

EDIT