1

@slhck provided a useful answer to this similar question.

Downloading multiple files with wget and handling parameters

This code was provided to read urls from a text file filelist, then run a loop to download each.

    #!/usr/bin/env bash 
    while read line 
      do   
        wget -c --load-cookies cookies.txt $line -O ${line##*/} 
    done < filelist

 

As I'm not familiar with shell scripts, I am wondering if you could download multiple xml files by:

  • creating a simple comma seperated text file filelist
  • read from:

filelist

renamedfile.xml, url-01.php
renamedFileDifferent.xml, url-02.php
specificFileRename.xml, "url-03"
newFilename.xml, "url-04"

  • read through each line
  • split the line into newfile, remoteFile

  • And run:  

    wget -O newfile remoteFile >/dev/null 2>&1

 

Is it possible to write this in a shell script? Can you provide an example?

Community
  • 1
rrrfusco
  • 145

1 Answers1

1

To get the text from before the comma you use ${line/,*}.

(What this actually does is replace ",*" or all the text from the comma onwards with empty string - leaving only the part of the string before the comma)

To get the text from after the comma you use ${line/*,}.

So the full command would be:

while read line
  do
    wget -O ${line/,*} ${line/*,} >/dev/null 2>&1 
done < filelist

or, on one line:

while read line; do echo wget -O ${line/,*} ${line/*,} >/dev/null 2>&1; done < filelist

In Windows (assuming you've installed wget, from http://gnuwin32.sourceforge.net/packages/wget.htm, and set your path correctly), it would be:

for /f "delims=, tokens=1*" %a in (filelist) do wget -O %a %b
Luke
  • 1,155