5

I want to run 2 commands on a piped-in input and want to print (to stdout) the output of both.

Each command is a combination of grep, sed and awk.

Both these commands must reside in a single .sh file.

Sample commands:

cat mult_comm.sh       
sed 's/World/Boy/g'|grep Boy ; grep World

# Input
cat input.log
Hello World

# This command HAS to work EXACTLY like this
cat input.log | bash mult_comm.sh

Expected output

Hello Boy
Hello World

Actual output

Hello Boy

I tried using tee

cat mult_comm.sh
tee >(sed 's/World/Boy/g'|grep Boy) | grep World

But this gives only 

Hello World

I can modify the .sh file as I want but the piped command can't be changed. Any ideas?

This is similar to OS X / Linux: pipe into two processes? and Pipe output to two different commands, but I can't figure out how to use named pipes inside the script.

Community
  • 1
  • 1
SANDeveloper
  • 560
  • 2
  • 7
  • 22
  • Is it cheating to do it with just sed? `sed -n 'h;s/World/Boy/g;/Boy/p;x;/World/p' < input.log` – pilcrow Aug 01 '13 at 13:30

2 Answers2

15

When you execute

tee >(some_command)

bash creates a subshell to run some_command. The subshell's stdin is assigned to the reading half of a pipe. bash leaves the name of this pipe on the command line, so that tee will pump its input into the pipe. The subshell's stdout and stderr are left unchanged, so they are still the same as tee's.

So, when you execute

tee >(some_command) | some_other_command

Now, bash first creates a process to run tee, and assigns its stdout to the writing half of a pipe, and another process to run some_other_command, with its stdin assigned to the reading half of the same pipe. Then it creates another process to run some_command, as above, assigning its stdin to the reading half of another pipe, and leaving its stdout and stderr unchanged. However, stdout has already been redirected to some_other_command, and that's what some_command inherits.

In your actual example,

tee >(sed 's/World/Boy/g'|grep Boy) | grep World

we end up with:

                  -->  sed 's/World/Boy/g' -->  grep Boy --
                 /                                         \
input --> tee --<                                           \
                 \                                           \
                  ----------------------------------------------> grep World

In one of the questions linked in the OP, there is a (non-accepted but correct) answer by F. Hauri, which I've adapted here:

echo Hello World |
((tee /dev/fd/5 | grep World >/dev/fd/4) \
           5>&1 | sed 's/World/Boy/' | grep Boy) 4>&1

It takes a little practice to read bashisms like the above. The important part is that

( commands ) 5>&1

Creates a subshell (( )) and gives that subshell an fd numbered 5, initially copied from stdout (5>&1). Inside the subshell, /dev/fd/5 refers to that fd. Within the subshell, it is possible to redirect stdout, but that will happen after stdout is copied to fd5.

Community
  • 1
  • 1
rici
  • 234,347
  • 28
  • 237
  • 341
  • This makes sense except that I am not so hot with file descriptors. Any idea why I would be getting:tee: /dev/fd/5: No such file or directory – SANDeveloper Aug 01 '13 at 02:38
  • @SANDeveloper: You need to do it exactly as shown. `(tee /dev/fd/5 ... ) 5>&1`. Here the `( ) 5>&1` creates a subshell with fd5 made a duplicate of fd1. Inside that subshell, you can use /dev/fd/5. Similarly with /dev/fd/4, which is generated with `(...) 4>&1` – rici Aug 01 '13 at 02:41
1

You can use pee(1) – in Debian/Ubuntu it's available in the package moreutils.

Usage for your example, somewhat more readable than the magic redirection

echo Hello World | pee 'grep World' 'sed "s/World/Boy/" | grep Boy'
Morten
  • 634
  • 5
  • 13