Loop over files with extension does not work

Question

I have several Name*.txt files in /home/user/my/path/to/my/data/, among other files with different extensions. I would like to loop over them, then use the individual file names in the code, therefore common solutions like this won't work, since the varible '$f', within each loop, stores the whole path together with the file name. I need them separately, to perform something like the "example taks" below. My attempts:

Attempt #1:

#!/bin/bash

datapath="/home/user/my/path/to/my/data/"
outpath="/home/user/my/path/to/my/outputs/"

for f in $(ls $datapath"Name*.txt"); do
        echo $f
        ...
        cp $datapath$f $outpath"example_task"${f:0:10}
done

This didn't work: ls: cannot access /home/user/my/path/to/my/data/Name*.txt: No such file or directory. Although running ls /home/user/my/path/to/my/data/Name*.txt on the terminal works perfectly fine. I can't understand why.

Attempt #2:

#!/bin/bash

datapath="/home/user/my/path/to/my/data/"
outpath="/home/user/my/path/to/my/outputs/"

for f in $datapath"Name*.txt"; do
        echo $f
        ...
        cp $datapath$f $outpath"example_task"${f:0:10}
done

Here, each $f contains the full list of files ls Name*.txt would normally return, and not one at a time as one would expect.

How do I do this? Any suggestions will be much appreciated.

How about `for f in "$datapath"/*.txt ; do echo "$f" ; done` — James Brown, Oct 08 '19 at 09:34
The first attempt is attempting to parse `ls` which is [not a good idea](https://unix.stackexchange.com/questions/128985/why-not-parse-ls-and-what-to-do-instead). Your second attempt is close, just drop the double quotes. — JNevill, Oct 08 '19 at 09:35
Well, remove it from one or the other, you could leave it too. — James Brown, Oct 08 '19 at 09:40
@JNevill that works for all `*.txt`. In my case, however, I actually need to loop over `Name*.txt`. Therefore, removing the double quotes mixes the datapath variable name with the file name: `$datapathName*.txt` won't work. I will edit the question to include this contraint. — ouranos, Oct 08 '19 at 09:40
@JamesBrown this causes the same problem as Attempt #2, each '$f' contains the full list of files that satisfy the `Name$.txt`, and not one at a time. Would you know why? — ouranos, Oct 08 '19 at 09:43
@JNevill that worked like a charm. Thank you! Would you like to wirte an answer to formalize the stackoverflow workflow? :) — ouranos, Oct 08 '19 at 09:46
To the moderators, is there any negative aspect in this question to justify a down vote? — ouranos, Oct 08 '19 at 11:19

score 1 · Accepted Answer · answered Oct 08 '19 at 09:44

Maybe I am on the wrong path here but this worked for me.

#!/bin/bash

datapath="/home/user/my/path/to/my/data/" 
outpath="/home/user/my/path/to/my/outputs/"

cd $datapath

for f in ./*.txt; do
        file=$(echo $f | cut -d '/' -f 2)

        echo $file
        ...
        cp $f $outpath"example_task"$file 
done
cd

Loop over files with extension does not work

1 Answers1