Read line by line in Bash script

Question

I want to do the following: Read a file line by line and use the line as a parameter.

FILE="cat test"
echo "$FILE" | \
while read CMD; do
echo $CMD
done

But when I do echo $CMD, it just prints cat test.

Answer 1

The best way to do this is to redirect the file into the loop:

# Basic idea. Keep reading for improvements.
FILE=test

while read CMD; do
    echo "$CMD"
done < "$FILE"

A redirection with < "$FILE" has a few advantages over cat "$FILE" | while .... It avoids a useless use of cat, saving an unnecessary child process. It also avoids a common pitfall where the loop runs in a subshell. In Bash, commands in a | pipeline run in subshells, which means variable assignments are lost after the loop ends. Redirection with < doesn't have that problem, so you could use $CMD after the loop or modify other variables inside the loop. It also, again, avoids unnecessary child processes.

There are some additional improvements that could be made:

• Add IFS= so that read won't trim leading and trailing whitespace from each line.
• Add -r to read to prevent backslashes from being interpreted as escape sequences.
• Lower-case CMD and FILE. The Bash convention is that only environmental and internal shell variables are uppercase.
• Use printf in place of echo which is safer if $cmd is a string like -n, which echo would interpret as a flag.

file=test

while IFS= read -r cmd; do
    printf '%s\n' "$cmd"
done < "$file"

Answer 2

What you have is piping the text "cat test" into the loop.

You just want:

cat test | \
while read CMD; do
    echo $CMD
done

Answer 3

xargs is the most flexible solution for splitting output into command arguments.

It is also very human readable and easy to use due to its simple parameterisation.

Format is xargs -n $NUMLINES mycommand.

For example, to echo each individual line in a file /tmp/tmp.txt you'd do:

cat /tmp/tmp.txt | xargs -n 1 echo

Or to diff each successive pair of files listed as lines in a file of the above name you'd do:

cat /tmp/tmp.txt | xargs -n 2 diff

The -n 2 instructs xargs to consume and pass as separate arguments two lines of what you've piped into it at a time.

You can tailor xargs to split on delimiters besides carriage return/newline.

Use man xargs and google to find out more about the power of this versatile utility.

Answer 4

Do you mean to do:

cat test | \
while read CMD; do
echo $CMD
done

Answer 5

If you want to use each of the lines of the file as command-line params for your application you can use the xargs command.

xargs -a <params_file> <command>

A params file with:

a
b
c
d

and the file tr.py:

import sys
print sys.argv

The execution of

xargs -a params ./tr.py

gives the result:

['./tr.py', 'a', 'b', 'c', 'd']

Answer 6

while read CMD; do
    echo $CMD
done  << EOF
data line 1
data line 2
..
EOF

Answer 7

The correct version of your script is as follows;

FILE="cat test"
$FILE | \
while read CMD; do
echo $CMD
done

However this kind of indirection --putting your command in a variable named FILE-- is unnecessary. Use one of the solutions already provided. I just wanted to point out your mistake.