2

I came across the answer to How can I send the stdout of one process to multiple processes using (preferably unnamed) pipes in Unix (or Windows)? and was playing around with grep when I noticed a strange issue. The following works as I'd expect:

$ while [ 1 ]; do sleep 1 ; echo tick; done | tee >(grep -o ti) >(grep -o ic) >(grep tick) >/dev/null
ic
ti
tick
ic
tick
ti
^C

All three grep commands are applied to the output of the loop.

However, if I pipe the output of tee into a grep (instead of redirecting it to /dev/null), the greps on the files stop working:

$ while [ 1 ]; do sleep 1 ; echo tick; done | tee >(grep -o ti) >(grep -o ic) | grep tick
tick
tick
^C

Why does this happen?

Community
  • 1
  • 1
Christopher Neylan
  • 8,018
  • 3
  • 38
  • 51

1 Answers1

3

I'll hazard a guess...

tee >(grep -o ti) >(grep -o ic) >(grep tick) >/dev/null
tee >(grep -o ti) >(grep -o ic) | grep tick

In the first command, the arguments are processed left-to-right, so grep -o ti has its output going to standard output, and so do the other two process substitutions, and then the standard output of tee is redirected to /dev/null (but the processes are already launched with the original standard output).

In the second command, the pipe to grep tick is already set up before the shell works on >(grep -o ti), so the standard output of the grep goes down the pipe. The ti and ic lines are not selected by the grep tick command. If you changed that to grep i, you'd probably see the ti and ic lines too.

I had to adjust the command to:

while true; do echo tick; done |
tee >(grep -o ti) >(grep -o ic) | grep i | grep -v tick

to see the ti and ic commands. This is related to buffering with standard I/O; the output from the grep commands is going to a pipe, so it is fully buffered, and the outputs appear in batches. Unless I dropped the sleep 1, it would have taken far too long to see the alternative outputs.

A still better command for viewing the output was:

while true; do echo tick; done |
tee >(grep -o ti) >(grep -o ic) | grep i | uniq

The output contained lines like:

ictick
tick
ic
i
ti
tick
ic
tick
ti
ttick
ic
itick
tick
ic
i
ti
tiic
ictick
tick

The moral of the story is probably to ensure that the outputs of each process substitution go to a known file.

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
  • so, to be clear, the output of the loop and the output of the two greps inside the file redirections are all going into the `grep tick`? – Christopher Neylan Apr 19 '13 at 12:15
  • Yes, that seems to be what is happening. – Jonathan Leffler Apr 19 '13 at 12:37
  • yes, i see this now. compare the output of `while true; do echo tick; done | tee >(grep -o ti) >(grep -o ic) | grep -o tick | uniq` and `while true; do echo tick; done | tee >(grep -o ti) >(grep -o ic) | grep tick | uniq`. very interesting, thank you! – Christopher Neylan Apr 19 '13 at 13:34