Bash: While loop ignoring IFS on first line when inside loop

Question

I have a while loop that reads from the below text file:

hashicorp/aws/4.27.0
hashicorp/aws/4.26.0
hashicorp/aws/4.11.0
...

The while loop that reads from it is as follows:

while read -r owner provider version; do \
    IFS=/
    echo $owner
done < providers.txt

output:
    hashicorp aws 4.27.0
    hashicorp
    hashicorp

It for some reason seems to not use the set IFS on the first line.

However, I can get the correct result by moving the IFS:

while IFS=/ read -r owner provider version; do \
    echo $owner
done < providers.txt

output:
    hashicorp
    hashicorp
    hashicorp

But I don't understand why in the first instance it doesn't use the IFS correctly as IFS=/ is still being set to before the echo statement. Can someone please explain?

Same could be done in this case reading the entire line and applying variable expansion `line='hashicorp/aws/4.11.0'; echo "${line%%/*}"`. — LMC, Aug 23 '22 at 16:48
Assuming `IFS` wasn't modified before run of this piece of code, `read` uses the default value of the `IFS`, so word splitting isn't performed based on the `/` character on the first line. — M. Nejat Aydin, Aug 23 '22 at 17:33

score 3 · Accepted Answer · answered Aug 23 '22 at 17:36

It's because the first version doesn't set IFS to "/" until after the first line has been read and split. When and where you set a variable determines what it affects, and it's important to understand the differences.

Walking through what gets executed might help. When the script reaches the while loop, it first executes the while clause:

while read -r owner provider version

...which reads the first line from the file. Since IFS still has its default value (space + tab + newline) at this point, the line is split based on that (i.e. not split at all), so the entire line goes in the owner variable.

Next, since the while clause succeeded, the body of the while loop executes:

    IFS=/
    echo $owner

...this sets IFS and then echoes $owner. Since $owner is not double-quoted, and by the time that runs IFS has been set to "/", the value gets word-split based on "/", so what's effectively executed is echo "hashicorp" "aws" "4.27.0", which sort-of has the effect of turning the "/" characters in the variable into spaces. This sort of weird parsing effect is why you should almost always double-quote variable references.

Next, the loop runs again. This time IFS has been set to "/", so the read command will split the line based on that, and from this point on it'll behave as you expect.

Now, compare that to the second version:

while IFS=/ read -r owner provider version; do

Here, the assignment to IFS isn't a separate command, it's a prefix to the read command. This means it always applies to the read command, and also makes it apply only to the read command. So it applies every time read runs, and also it doesn't affect other things later, like any unquoted variable references. Setting IFS to a nonstandard value can cause trouble, and restricting it to a single command like this is a good way to keep it from causing trouble later.

Lessons: 1) Double-quote your variables, and 2) set IFS as a prefix to specific commands when possible (when that's not possible, set it before the command it needs to affect, and then set it back to normal as soon as possible afterward).

score 1 · Answer 2 · answered Aug 23 '22 at 16:23

The IFS variable in bash controls how words are split by the shell. By default this is space, tab, and newline. The reason that you set IFS is so when wordspliting happens on your read call, the words are split by / instead of spaces. This variable is not relevant to the echo call because the owner variable already contains just the word hashicorp.

Bash: While loop ignoring IFS on first line when inside loop

2 Answers2