1

On linux, with Python 3.8 I'm trying to process text data line by line read from a named pipe (created with mkfifo). The reading takes place on a thread (main thread runs an asyncio event loop).

Problem is that the below code blocks at open() indefinitely when the other process - that sends the data - is not running, however I need to stop the thread when there is no data coming from the pipe after a certain amount of time.

Initally I tried with select.select() before I realised that it blocks at the open(), not at readline(). Then I tried with signal.signal setting an alarm, but signalling only works in the main thread, this should run in a thread pool. The best thing would be either to make the open() non blocking somehow, or before open() detect whether the other half of the pipe is open, also in a non-blocking fashion.

while True:
    with open(pipe_file_path, 'r') as pipe_file:
        _log.debug("Reading log")
        while True:
            line = pipe_file.readline(768)
            if line == '':
                break
            process_line(line.rstrip()) # arbitrary processing
    _log.debug("Pipe closed")

The whole thing is in an endless loop (1st line), since I want to re-open the pipe immediately when the other process closes it briefly (sometimes that happens), e.g. when it re-parses its configuration. So the ultimate goal is: try to read from the pipe when it can be opened within T time, otherwise stop waiting.

Z4-
  • 1,851
  • 14
  • 17
  • 1
    See https://stackoverflow.com/questions/14345816/how-to-read-named-fifo-non-blockingly – Barmar Sep 20 '20 at 02:09
  • @Barmar thank you, `fd = os.open(pipe_file_path, os.O_RDONLY | os.O_NONBLOCK)` then `os.read(fd)` does the trick. I'll try to experiment with `select.poll()` as well. – Z4- Sep 20 '20 at 02:47

0 Answers0