0

It seems whenever I test this on the shell it works but when I'm running it from a script it won't.

Shell:

fips=55555
echo https://www2.census.gov/geo/tiger/TIGER2016/ROADS/tl_2016_$fips'_'roads.zip
>>https://www2.census.gov/geo/tiger/TIGER2016/ROADS/tl_2016_55555_roads.zip

I'm trying to loop through some data in a text file like this:

enter image description here

And parse it/input into a curl url:

But the output has the beginning of the string overwritten like this:

./readFipsData.txt
>>_roads.zipw2.census.gov/geo/tiger/TIGER2016/ROADS/tl_2016_48001

Note: I am using Git Bash for Windows if that makes any difference.

cat ./testFipsData.txt |
while read line
do
    counter=0
    for col in $line
    do
    if [ $counter == 0 ]
    then
        state=$col
    elif [ $counter == 1 ]
    then
        county=$col
    elif [ $counter == 2 ]
    then
        fips=$col
    fi
        ((counter++))
    done
#curl -k -o ./$county.zip "https://www2.census.gov/geo/tiger/TIGER2016/ROADS/tl_2016_$fips_roads.zip"
echo https://www2.census.gov/geo/tiger/TIGER2016/ROADS/tl_2016_$fips'_'roads.zip
done
cevondanady
  • 111
  • 3
  • 1
    The input file probably contains MSWin line endings. Remove `$'\r'` from the input. – choroba Sep 29 '17 at 22:51
  • 2
    Don't use a pipe, use input redirection `done < testFipsData.txt` – Barmar Sep 29 '17 at 22:52
  • @choroba No, the problem is that he's setting the variable in a subshell because of the pipe. – Barmar Sep 29 '17 at 22:52
  • Never mind, I misread the code because of poor indentation. – Barmar Sep 29 '17 at 22:54
  • @choroba If he's running this on Windows, shouldn't it read properly? That's usually a problem only if you transfer a Windows file to Unix. – Barmar Sep 29 '17 at 22:56
  • I've never worked with `bash` on Windows except from `cygwin` which behaves exactly like *nix in this regard. – choroba Sep 29 '17 at 22:59
  • Can put `line=${line%$'\r'}` in the loop to moot the issue if it's present. – Charles Duffy Sep 29 '17 at 23:00
  • Similar to what @CharlesDuffy is suggesting - `for col in ${line%$'\r'}`. – alvits Sep 29 '17 at 23:01
  • @cevondanady, btw, `==` isn't guaranteed to work inside `[ ]`; bash supports it as an extension, but many versions of `/bin/sh` only support `=` (which is the POSIX-specified string comparison operator). You might thus want to get used to using `[ "$counter" = 0 ]` -- or, if you want to use bashisms, `if (( counter == 0 ))`, which gives you a much wider array of math operations with C-style syntax. – Charles Duffy Sep 29 '17 at 23:01
  • 2
    It would have been better to use `while read state city zip`. – alvits Sep 29 '17 at 23:03
  • That's what I get for not reading context. Absolutely so. :) – Charles Duffy Sep 29 '17 at 23:03
  • 2
    @alvits, ...maybe `while IFS=$' \t\r' read -r state city zip _`, to ignore any extra fields and trim the carriage returns, and not mangle backslash literals. – Charles Duffy Sep 29 '17 at 23:04
  • 2
    though... what does that data do with city names that have spaces? I wonder if only tabs and carriage returns (but not regular spaces) should be in IFS. – Charles Duffy Sep 29 '17 at 23:08
  • @cevondanady, an embedded image of how a terminal renders your data doesn't give us much that's useful about what that data's actual format is. It doesn't tell us if the whitespace is tabs or spaces, it doesn't tell us if there are CRLFs or LFs as newlines, etc. If you have GNU cat, consider posting a few lines of `cat -A yourfile` output **as text** [edit]ed into the question, and literally formatted by selecting those lines (all at once!) and clicking `{}`, or adding a four-space indent. – Charles Duffy Sep 30 '17 at 00:24

1 Answers1

0

Thanks to all of your suggestions here's what worked:

cat ./testFipsData.txt | 
while read state county fips
do
fips=${fips%$'\r'}
curl -k -o ./$county.zip https://www2.census.gov/geo/tiger/TIGER2016/ROADS/tl_2016_$fips'_'roads.zip
done
cevondanady
  • 111
  • 3