1

I'm trying to pipe a pair of Linux commands using python.
Example in Linux terminal:

ls -l | less

Here's a pseudo-code of what I do:

  1. I define a pipe.
  2. Fork and make a child for the parent process.
  3. Set the stdout file descriptor number to the write-end of the pipe.
  4. Execute the command and Exit.
  5. Fork another process.
  6. Set the stdin file descriptor number to the read-end of the pipe.
  7. Execute the command.

But it's not working.

 child1 = os.fork()
    if child1 < 0:
        raise fork_exception
    elif child1 == 0:
        oread, owrite = os.pipe()
        os.close(oread)
        os.dup2(owrite, sys.stdout.fileno())
        os.dup2(owrite, sys.stderr.fileno())
        os.execvp(command[0][0], args=command[0])
        child2 = os.fork()
        if child2 < 0:
            raise fork_exception
        elif child2 == 0:
            os.close(owrite)
            os.dup2(oread, sys.stdin.fileno())
            os.execvp(command[1][0], command[1])
            sys.exit()
        os.waitpid(child2, 0)
        sys.exit(0)
    os.waitpid(child1, 0)

command is a nested list of piped commands.

command = [["ls", "-l"],["less"]]

p.s 1: I tried making the child2 and child1 two separate children of one parent, but that didn't work either.

p.s 2: I made a temporary file and used it as pipe, and that worked well. But using os.pipe(), It's not working, and I can't find the problem with my code.

Note: I don't want to use os.subprocess() or any other function or system call. I want to make this code, which is implemented using os.fork() and os.pipe(), running.

Mehrdad Heshmat
  • 91
  • 5
  • There is no such thing as a "Linux command". Both `ls` and `less` are simply programs. Using a pipe `|` in most shells you can forward one program's output into another program's input. This is called output redirection and also (hint!) a keyword for research. – Ulrich Eckhardt Nov 17 '21 at 16:21
  • 1
    The necessary logic isn't meaningfully Python-specific -- C questions like https://stackoverflow.com/questions/30822887/program-stuck-on-pipe-exec-ls-grep-sort address the core issues. – Charles Duffy Nov 17 '21 at 16:23
  • I'm not trying to redirect outputs, that's done using ">" in the terminal of the UNIX-based system. But that's not my intention. I'm trying to establish a pipe between two processes. – Mehrdad Heshmat Nov 17 '21 at 16:27
  • @Charles Duffy Yes, I've seen many C examples of piping, and like you said it's the same logic. But I don't know what's going wrong here. I asked my question in Python section because it's what my code is in, and there might be slight differences in implementations of these features. Like the difference between "os.pipe()" and "pipe(pipe_fd1)" – Mehrdad Heshmat Nov 17 '21 at 16:32
  • 1
    The mistakes I see here looking at your code are all things that would be mistakes in C as well. For example, you can't `wait()` until **both** sides of your pipeline are started up (because all parts of a pipeline run in parallel; it's not one-side-then-the-other), and you should be passing specific PIDs to the `wait()` call (so use `os.waitpid()`, not just `os.wait()`). You also should be closing pipeline components in the parent process so only the children have open handles; otherwise, the parent having a handle on the write side will prevent the read side from ever getting an EOF. – Charles Duffy Nov 17 '21 at 16:49
  • and also while you are doing dup2 of writefd shouldn't it be in reverse order, you want to copy data to your fd that is written into standard output `os.dup2(sys.stdout.fileno(),write0)` – Gaurav Nov 17 '21 at 16:58
  • Is `less` really what you need here? Given its nature, I’d expect problems with that particular program. Would `cat` or `egrep` serve as examples too? – JL Peyret Nov 17 '21 at 17:17
  • @Gaurav dup2 doesn't copy the data. it duplicates the file descriptor. I believe that the order is right, but I cant find the problem. – Mehrdad Heshmat Nov 22 '21 at 22:57
  • @JLPeyret No, it's just an example of piping the commands. I'm trying to implement a pipe functionality for my shell... – Mehrdad Heshmat Nov 22 '21 at 22:58
  • @MehrdadHeshmat Yes what i mean was same, you should duplicate the stdout to your fd, because by default any command will sending output to stdout not to your fd – Gaurav Nov 23 '21 at 03:32

0 Answers0