2

I am trying to interact with an external program in C using pipe, fork and exec. I want to force the external program to perform unbuffered I/O. Here's a relevant snippet from my code so far:

...
pid = fork();
if (pid == (pid_t) 0)
{
    /* child process */

    /* make pipe connections to standard streams */             
    dup2(wpipe[0], STDIN_FILENO);
    dup2(rpipe[1], STDOUT_FILENO);

    /* close pipe endings */
    close(wpipe[0]); close(rpipe[0]);
    close(wpipe[1]); close(rpipe[1]);

    /* unbuffered I/O */
    setvbuf(stdin, NULL, _IONBF, BUFSIZ);
    setvbuf(stdout, NULL, _IONBF, BUFSIZ); 

    if (execl("path/to/some_binary", "path/to/some_binary", (char *) NULL) == -1)
    {
        fprintf(stderr, "exec failed\n");
        return EXIT_FAILURE;
    }   
    return EXIT_SUCCESS;
}
...

This does not work because streams do not survive across exec calls. So using setvbuf to force unbuffered I/O does not work, because the program image (some_binary) creates stdin and stdout streams of its own, and does not use the streams that I called setvbuf on.

The program does work when I re-build some_binary after adding the setvbuf calls in its code. But how can this be done if you do not have any control over the binary you are passing to exec? How can this be made to work for, say, a unix command like cat or ls?

alk
  • 69,737
  • 10
  • 105
  • 255
prvnsmpth
  • 195
  • 4
  • 15
  • Check the `unbuffer` command that comes with [expect](http://expect.sourceforge.net/). – Brian Dec 13 '13 at 16:46
  • Why do you ask?? What is the use-case???? – Basile Starynkevitch Dec 13 '13 at 16:51
  • When doing two-way communication with a sub-process using two pipes. With I/O buffering, read operations in the parent process get blocked simply because the output has been buffered by the child sub-process. – prvnsmpth Dec 13 '13 at 16:55
  • What *exactly* is the program running (after `execve`) in the *child* process? – Basile Starynkevitch Dec 14 '13 at 13:49
  • related: [There are existing utilities that can do it for you](http://stackoverflow.com/a/20509641/4279). Ignore Python-specific code or reimplement it in C e.g., there is a select loop over pseudo-tty. – jfs Dec 14 '13 at 18:41

2 Answers2

1

You cannot do what you want in the general case (unbuffering after execve(2) of arbitrary executables...)

Buffering is done by code (e.g. by some libc code related to <stdio.h>). And the code is defined by the program being execve-ed.

You might perhaps play with LD_PRELOAD tricks which might call setvbuf(stdin, NULL, _IONBF, BUFSIZ); after the execve (but before the main....); but this would work only with dynamically linked executables.

Perhaps using some constructor function attribute in some initialization function of your LD_PRELOAD-ed shared object might sometimes do the trick. Or redefine printf, fopen, .... in that shared object...

addenda

You commented that you do :

two-way communication with a sub-process using two pipes.

Then your approach is wrong. The parent process should monitor the two pipes, probably with a multiplexing call like poll(2), then (according to the result of that multiplexing) decide to read or to write to the child process. In reality, you want some event loop: either implement a simple event loop yourself (with e.g. poll [iteratively called many times] inside a repeated loop) or use some existing one (see libevent or libev, or the one provided by some toolkit like GTK or Qt etc...)

You might also multiplex with select(2) but I recommend poll because of the C10K problem

You won't lose your time by reading Advanced Linux Programming ...

See also this answer to your next related question.

Community
  • 1
  • 1
Basile Starynkevitch
  • 223,805
  • 18
  • 296
  • 547
  • Thank you for the addendum! Is the poll function available in GNU C? Why does it not appear in the documentation of GNU C? – prvnsmpth Dec 13 '13 at 17:08
  • What do you call GNU C? Is it [GCC](http://gcc.gnu.org/) the compiler (which is unrelated to your question) or [Glibc](http://www.gnu.org/software/libc/) (a.k.a. GNU libc) the standard C library? `poll` is documented in POSIX [poll](http://pubs.opengroup.org/onlinepubs/009695399/functions/poll.html) and in the man pages!! And GNU libc does not document all of POSIX (but assumes it is known). BTW, GNU libc documents [select](http://www.gnu.org/software/libc/manual/html_node/Waiting-for-I_002fO.html#Waiting-for-I_002fO) -which is an old-fashion way to multiplex. – Basile Starynkevitch Dec 13 '13 at 17:09
  • I meant glibc. I couldn't find the poll function documented on this page: http://www.gnu.org/software/libc/manual/html_mono/libc.html – prvnsmpth Dec 13 '13 at 17:15
  • I tried to implement polling, but again ran into problems because of buffering - http://stackoverflow.com/questions/20581466/using-poll-function-with-buffered-streams – prvnsmpth Dec 14 '13 at 09:10
  • You need an *event loop* the `poll` should be *repeated* inside a loop... (with *different* actions when you get *readable* and *writable* file descriptors given by `poll`) – Basile Starynkevitch Dec 14 '13 at 11:39
-1

You might try to pause the task, then modify its image. You can do it via gdb or the /proc filesystem. In particular, you have the access to all fds the task created via the /proc filesystem.

UPDATE:

If the buffering is done by glibc, just override the relevant function in glibc, by defining your own function, compiling it into a shared library and loading with the program.

haael
  • 972
  • 2
  • 10
  • 22