2

So, I was looking to take the output of unzip -Z1 into an array and found this answer; their first option, using mapfile and process substitution WITH input redirection, works a charm.

But then I thought, "wait, process substitution," which creates a file descriptor from the stdout, "then use input redirection on that to put the contents into stdin?"
Is that not equivalent to just piping?

Apparently no. No it isn't. Here I try to put the contents of ls into a variable, but using a pipe: à la ls | mapfile -t test
enter image description here Nada.

But, if I follow that answer to a T: id est mapfile -t test < <(ls)
enter image description here Voilà.

But why? diff can't tell the difference. At least between their content.
enter image description here

I couldn't find anything inherently special about < <(), other than it creates a file descriptor. Using that as a theory, I tried a HEREDOC, which makes a file descriptor too. It worked:

mapfile -t test2 << END && printf '%s ' "${test2[@]}"
this
is
an
array
END

But not if it's laundered through stdin via cat, try:

cat << END | mapfile -t test3 && printf '%s ' "${test3[@]}"
this
is
an
array
END

So, my questions.
That HEREDOC discovery/file descriptor theory shouldn't matter for mapfile as its manuals all say it uses "standard input", right?
If 'standard input' somehow means it has to be a fancier descriptor; why? Why is it unable to use a simple read-once input stream to generate the array?
And finally if both of those are answered/on the right track, why didn't the command fail when it expected an fd? You'll see my terminal has 0 printed a bunch (and we use && for the last examples), that's $? on my prompt, so there was no error reported by the builtin.

I tried this on Fedora 30 and RHEL 6, so I don't think it's a bug.

Hashbrown
  • 3,338
  • 4
  • 39
  • 51

1 Answers1

3

It works; it's just that the results disappear immediately afterwards.

The difference is that foo | mapfile is a pipeline consisting of two new processes – each element, be it an external program or a shell builtin, is run in a subshell. So although mapfile still does its job it has no way of transferring the result to the parent shell process.

For comparison, you would get the same result with (mapfile < foo.txt) or ls | while read line; do ... done or even (foo=bar); echo $foo. In all these cases, if you compare $BASHPID and $$, you'll see different values – the former shows the subshell process PID being different from the main shell PID.

When using a redirection, the mapfile command runs in the main shell process and can alter shell variables correctly. It doesn't much care if it's taking redirection from a real file, or from a <<-generated temporary file, or from a <()-generated magic file – in this case, the shell handles that as a separate step and doesn't cause a pipeline to be built. (The inner command of <() or $() is a subshell, of course.)

Note that the variable can actually be used in the subshell – it just cannot be transferred upwards. So for one-shot operations, the 2nd part of pipeline can be quite elaborate:

unzip -l | {
    mapfile -t test
    for file in "${test[@]}"; do
        echo "I got $file!"
    done
}

zipinfo | while read -r file; do
    echo "I got $file!"
done
grawity
  • 501,077