0

It seems that wget is not able to get asp files while it has no problem with html. I have wrote a script to wget some URLs and save them in csv format. The script is

wget -qO- http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KPAPALMY1&format=1 | sed 's+<br />++g' > $1.csv
mahmood
  • 23,197
  • 49
  • 147
  • 242
  • Can you post your whole script? Also: `wget`ing that page works fine for me. But the page the server returns isn't in CSV format - there are extraneous `
    ` tags thrown in, and each line seems to end with an unnecessary comma.     – Xavier Holt Oct 27 '12 at 13:58

1 Answers1

1

You have to put quotes around that URL! Left unquoted, Bash (or whatever shell you're using) sees the & as a request to run wget in the background - and the format parameter, which comes after the & is never sent to the server. Try this instead:

wget -qO- 'http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KPAPALMY1&format=1' | sed 's+<br>++g' > $1.csv

Note that I tweaked your sed command, too - the <br>s you're getting don't have the terminating slash. Also: Deleting those <br>s means that only every other line has data. Hopefully that won't be a problem, as deleting newlines with sed is a bit of a pain, but if it is, see this question for how to do it.

Hope that helps!

Community
  • 1
  • 1
Xavier Holt
  • 14,471
  • 4
  • 43
  • 56