C: popen reading from stdout blocks

Question

The manpage for popen says "reading from a "popened" stream reads the command's standard output".

However, I can't seem to get the subprocess output in the trivial program below. The "reader" parent process blocks on the read (whether using fgets or fread)

What am I missing?

Attaching to the pinger program with gdb shows it is looping and calling printf to output text. Just nothing detected by fgets on the parent's side...

PINGER.C

#include <string.h>
#include <stdio.h>
#include <unistd.h>

int
main(int argc, char **argv)
{
    int i = 0;
    while (1)
    {
        printf("stdout %d\n", i++);
        sleep(1);
    }
}

POPENTEST.C

#include <stdio.h>
#include <errno.h>
#include <stdlib.h>


int
main(int argc, char **argv)
{
    char *cmd = "./pinger";
    printf("Running '%s'\n", cmd);

    FILE *fp = popen(cmd, "r");
    if (!fp)
    {
        perror("popen failed:");
        exit(1);
    }

    printf("fp open\n");

    char inLine[1024];
    while (fgets(inLine, sizeof(inLine), fp) != NULL)
    {
        printf("Received: '%s'\n", inLine);
    }

    printf("feof=%d ferror=%d: %s\n", feof(fp), ferror(fp), strerror(errno));
    pclose(fp);
}

OUTPUT

$ ./popenTest
fp open

score 3 · Accepted Answer · answered Jun 08 '19 at 04:02

3

By default, C buffers writes to the stdout when stdout is not connected to a tty. This means that from the OS' perspective, the program has not written anything to stdout until either the buffer is full or you manually flushed the output:

#include <string.h>
#include <stdio.h>
#include <unistd.h> 

int
main(int argc, char **argv)
{
    int i = 0;
    while (1)
    {
        printf("stdout %d\n", i++);
        fflush(stdout);
        sleep(1);
    }
}

When connected to a tty, the stdout is automatically flushed on every newline. But this automatic flushing does not happen when the stdout is connected to a pipe.

answered Jun 08 '19 at 04:02

Lie Ryan

62,238
13
100
144

2

C also buffers write to stdout by default when it is connected to a tty, but it is line buffered rather than block buffered. – William Pursell Jun 08 '19 at 04:07
duh, yes of course. Thank you. In the real case, the source of the data is a 3rd party program I don't control, so buffering is inevitable. What is the buffer size? (or how to find out?) – Danny Jun 08 '19 at 04:34
@Danny: many programs accepts command-line flags to control buffering, that's usually the best way to do this. However, if you can't, then there are [a few hacks](https://stackoverflow.com/questions/3465619/how-to-make-output-of-any-shell-command-unbuffered) you can do. – Lie Ryan Jun 08 '19 at 04:38
Good idea. The actual program is `ffmpeg`, which has the `-flush_packets` flag. That should do the trick. Thanks. – Danny Jun 08 '19 at 05:17
Any ideas on finding errors in the child would be appreciated. See: https://stackoverflow.com/questions/56503915/c-c-detect-fault-in-popen-child-process – Danny Jun 08 '19 at 06:06

C: popen reading from stdout blocks

1 Answers1