Integrate Error Message in while condition Bash

Question

I want to extract from a website multiple files, but I don't want to do this manually. Instead I want to download these files until there is an error like: ERROR 404: Not Found.

For example I have on this website some files called:

file1.txt
file2.txt
.
.
.
file10.txt

and I use wget http:.... If I try to download file11.txt, which does not exists, I get the ERROR 404: Not Found Error.

If I try to do something like:

while [condition]
do
   NR=$((NR+1));
   wget ...
done

where NR is the variable that will change the number of the file.

What should I use as condition to stop the loop when it reaches a file that does not exist?

score 2 · Answer 1 · answered Sep 16 '22 at 09:53

Simply break if wget fails.

while IFS= read -r url; do
    wget "$url" || break
done <file.txt

Your attempt seems to succumb to the antipattern in Counting lines or enumerating line numbers so I can loop over them - why is this an anti-pattern?

The same thing can work if the URLs don't come from a file.

for((i=1; ;++i)); do
    wget "http://site.example/file$i.txt" || break
done

The "C-style" for loop syntax here is a Bash extension, and not portable to regular sh.

score 2 · Accepted Answer · answered Sep 16 '22 at 10:08

You can place the wget call directly in the condition position of the while loop, since it works based on the exit code of the command (it will loop as long as the exit code is 0):

declare -i counter=1

while wget "http://example.com/file${counter}.txt"; do
  counter+=1
done

The -i part in declare -i makes sure that a few lines below += means arithmetic addition and not string concatenation.

Integrate Error Message in while condition Bash

2 Answers2