6

In the following I create a background process and wait for it to complete. 

$ bash -c "sleep 5 | false"  &  wait $!
[1] 46950
[1]+  Exit 1                  bash -c "sleep 5 | false"
$ echo $?
1

This works and the prompt returns after 5 seconds.

However, wait returns an error if I use one more pipe after it. 

$ bash -c "sleep 5 | false"  &  wait $!  | true
[1] 49493
-bash: wait: pid 49493 is not a child of this shell
hbaba@mbp-005063:~/misc$ echo $?
0
hbaba@mbp-005063:~/misc$ ps -T -f
  UID   PID  PPID   C STIME   TTY           TIME CMD
980771313 49493 69771   0 12:56AM ttys056    0:00.00 bash -c sleep 5 | false
980771313 49498 49493   0 12:56AM ttys056    0:00.00 sleep 5
    0 49555 69771   0 12:56AM ttys056    0:00.01 ps -T -f

What is happening here?

I am using bash version GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin15)

I can reproduce the wait error every time. I think it has something to do with each pipe being a separate subshell. https://unix.stackexchange.com/a/127346/212862

Maybe the wait $! command looks for the child process in the wrong shell. The error message mentions the 49493 pid. That is indeed the right pid for the bash -c … Command. The ps -T shows that.

There are relevant questions q1 and q2. But in them there is no pipe usage after the wait built-in.

Update

I had a misunderstanding about operator precedence in bash between the & and |. @randomir pointed that out in his answer. Adding curly braces makes the wait wait on the previously backgrounded process. For example: 

{ bash -c "sleep 5 | false"  &  wait $! ; } | true

This does not return the same wait error.

Hakan Baba
  • 1,897
  • 4
  • 21
  • 37

2 Answers2

14

There are two key points to observe here:

  • wait (a shell built-in) can wait only (the shell's) children
  • each command in a pipeline runs in a separate subshell

So, when you say:

cmd & wait $!

then cmd is run in your current shell, in background, and wait (being the shell's built-in) can wait on cmd's PID, since cmd is a child of that shell (and therefore a child of wait).

On the other hand, when you say:

cmd & wait $! | cmd2

then cmd is still run in your current shell, but the pipe induces a new subshell for wait (a new bash process) in which cmd is not its child, and wait can not wait on its sibling (a child of its parent).

As an additional clarification of shell's grammar - the & operator (along with ;, && and ||) separates pipelines, forming lists. So, a list is a sequence of pipelines, and a pipeline is a sequence of commands separated by |.

That means the last example above is equivalent to:

cmd & { wait $! | cmd2; }

and not to this:

{ cmd & wait $! ; } | cmd2

which is equivalent to what you have expected.

randomir
  • 17,989
  • 1
  • 40
  • 55
  • Are you saying that in the second example, `wait` gets executed in the same subshell as `cmd2` ? – Hakan Baba Oct 15 '17 at 17:44
  • No. In the last example, `cmd` is executed in one shell, `wait` in second, and `cmd2` in third. Let me add that clarification to my answer. – randomir Oct 15 '17 at 18:06
  • 1
    Wauvv, thanks @randomir. I had a major misunderstanding regarding operator precedence. I was thinking that `&` and `|` had the reversed precedence compared to how they actually work. Thanks for realizing that mistake . – Hakan Baba Oct 16 '17 at 04:32
0

Bash manual:

Each command in a pipeline is executed as a separate process (i.e., in a subshell).

You can check this with:

$ echo "me1: $BASHPID"
me1: 34438
$ sleep 100 &
[1] 34989
$ echo "me2: $BASHPID, child: $!"
me2: 34438, child: 34989
$ true | echo "me3: $BASHPID, child: $!"
me3: 34991, child: 34989

The first and second echo (me1 and me2) are executed in the context of the top shell (34438). The sleep child process (34989) is one of its children. The third echo (me3) is executed in the context of a subshell (34991). The sleep process is not one of its children any more.

Renaud Pacalet
  • 25,260
  • 3
  • 34
  • 51
  • Let's say `bash -c "sleep 5 | false" & n="$!"` is spawned in subshell X. In subshell X, the `bash -c "sleep 5 | false"` is again spawned in the background in subshell Y. Then for the last pipe the `echo $n` is spawned in subshell Z. Does the $! variable in subshell X get modified because of the subshell Z ? – Hakan Baba Oct 15 '17 at 08:16
  • How about this command: `bash -c "sleep 5 | true" & n=$! ; echo "From first pipe $n" | cat && echo "From seconds pipe $n"` It prints `[1] 54035 From first pipe 54035 From seconds pipe 54035` – Hakan Baba Oct 15 '17 at 08:22
  • If I run your command, `bash -c "sleep 5 | false" & n="$!" | echo $n` bash tells me `$ bash -c "sleep 5 | false" & n="$!" | echo $n -bash: !": event not found` – Hakan Baba Oct 15 '17 at 08:24
  • Removing the quotes around $! removes that error. But then nothing is printed. Are you sure by the time you executed that command there was already no $n defined in your shell ? – Hakan Baba Oct 15 '17 at 08:25
  • Sorry, my "*proof*" was bogus. I fixed it. – Renaud Pacalet Oct 16 '17 at 02:21
  • What is your bash version? It behaves differently in my case. I am on mac os el capitan `GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin15)`. First of all `echo "$!"` results in `-bash: !": event not found`. So I removed the double quotes from your examples. For me `$BASHPID` is also empty so I used `$$`: The background job: `$ jobs -l [1]+ 3353 Running sleep 10000 &` First case: `$ echo me:\ $$,\ child:\ $! me: 69771, child: 3353` Second case `$ true | echo me:\ $$,\ child:\ $! me: 69771, child: 3353` – Hakan Baba Oct 16 '17 at 03:47
  • Strangely in my case `$ echo $$ | cat; echo $$ | cat; echo $$` prints the same 69771 pid 3 times. I am very interested in what I am missing here. – Hakan Baba Oct 16 '17 at 03:53
  • You must be using a bash higher than 4.0 https://unix.stackexchange.com/a/203475/212862 – Hakan Baba Oct 16 '17 at 03:55
  • I realize in my bash the `echo $$` does not work as I expected. For example: `$ ( echo $$ && echo $PPID && sleep 10;) & true | cat` prints `[3] 7475 69771 69749` This is very strange. The pid of the background process is 7475 but `echo $$` in it prints 69771, which is the top level bash in that terminal – Hakan Baba Oct 16 '17 at 04:03
  • @HakanBaba I am also on Mac OS (Yosemite) but I replaced the default bash by the MacPort more recent version. I will not try to show you another proof that would work with your version of bash but the principle remains the same: commands in a pipe are executed in subshells. – Renaud Pacalet Oct 16 '17 at 04:38