4

I am writing width-memory and performance-sensitive multithread server. Therefore, I need the alternative for standard system() and popen() calls that does not use fork() that clone all process memory that usually takes too much time.

Seems, need to use vfork() then execve() to made it.

Can somebody help me with two functions:

  • Replacement for system() call. Example of behavior: one thread call the function to execute eg touch filename and called thread waiting until execution end. (all other threads must continue working)
  • Replacement for popen() call Example of behavior: the same behavior, but need to get output of command eg ls -flags (alternative for this code: Correct Code - Non-blocking pipe with popen )

thanks

Community
  • 1
  • 1
abrahab
  • 2,430
  • 9
  • 39
  • 64
  • 2
    `fork` is extremely optimized to avoid making a huge copy of all of the process's memory; usually it's implemented with copy-on-write so that no memory is actually copied unless it's absolutely necessary. Are you sure that `fork` is too slow? – templatetypedef Jun 11 '12 at 21:11
  • @templatetypedef yes, 3-4 days I am searching here for the solution why is my server time-to-time slow down significantly (14-30 seconds). And now I am 80% sure, that its memory issue. My server is ~100-300mb then called `ffmpeg` in popen() call that need `200 mb`, so, parent now is not less than `200 mb` and seems at popen() calls its slow down.. does not know who write to it, maybe its the system issue, because I does not see any delay at freebsd 7.4 (but server is quad core)... but on the old one (two cores) and freebsd 6.4 the delay is 14-30 seconds. – abrahab Jun 11 '12 at 21:17

2 Answers2

3

This is just a suggestion for how to approach your proposed solution. For the two specific actions you are asking about:

  • touch: you can achieve a similar effect by opening and closing the file you want to touch
    • be careful not to use the O_TRUNC flag on open
  • ls: it is a little more painful, because you will need to use dirent.h on POSIX systems, and walk the results of
    • opendir
    • readdir
    • make sure to call closedir when you are done

If you want to replace the system and popen calls with something equivalent using vfork, there is some care involved. The vfork call is a little tricky to use, because if the child does anything other than exec right after the call, you take the chance of corrupting the memory state of the parent.

For your replacement to system:

  • create a helper program to execute the string provided in the arguments
    • the helper program can either call system on the provided argument, or parse the command string and call exec
  • in your system replacement function, create an arg vector to call the helper program and passing in the program string you really want executed in as an argument to this program
  • after you call vfork, you immediately exec the helper program in the child
  • parent waits for child to complete

In your popen replacement:

  • create a helper program that accepts the stdout and stdin file descriptors as arguments, and the string of the command you want to execute
    • the helper program will dup the passed in descriptors to 0 or 1 (or both) as indicated by the arguments
    • the helper program can execute the string either with popen and proxy data between parent and child, or call exec after parsing the command string
  • in your popen replacement function, use pipe to create the stdout or stdin communication channel (as per the second popen function parameter) and create an arg vector to call the helper program, passing in the appropriate file descriptor number and command string as arguments
  • after you call vfork, you immediate exec the helper program in the child
  • you'll need a pclose replacement to reap the child process
jxh
  • 69,070
  • 8
  • 110
  • 193
  • with `vfork` and `exec` all threads of parent will hang and wait `for child to complete`. we must use `execve` to immediately continue parent after `vfork` to avoid server `hang`. but its not the replacement for `system` call, because we must wait only at the called thread. – abrahab Jun 11 '12 at 22:27
  • @abrahab: You can deal with that by having your helper program fork first before execing in the child of the helper, and let helper program exit right away. To get the blocking semantics of `system`, use a `pipe`, and the parent to the `system` replacement would read on the pipe fd after `wait` completes, when grand-child that execs command finishes and exits, it will automatically close the write end of the pipe. Similar technique for `popen` – jxh Jun 11 '12 at 22:48
  • possibly worth noting that the specific exec command used for system is well-known and works just as well with vfork as fork. execl("/bin/sh", "sh", "-c", cmd, NULL); – codetaku Jan 19 '17 at 19:38
  • @codetaku: Good point. Let me think how to word this, as I took replacing `system` to mean everything needed to be replaced (so, no invoking the shell). – jxh Jan 21 '17 at 02:03
0
Another way is to use the "fork()" with "execvp()" in child process. as mentioned by @jxh

int pid = fork();

if( pid == 0)
{
// In child process
  // redirect stderr/stdinput/stdoutput
  // prepare the execution command
  //use the exec() family of functions as per specific requirement.
}
else
{
// In parent process
  // check for open pipes and close the Std Output/Input/Error pipes.
}

The details references

https://www3.physnet.uni-hamburg.de/physnet/Tru64-Unix/HTML/APS33DTE/DOCU_004.HTM

SRM
  • 1
  • 2