How to loop over files in directory and change path and add suffix to filename

Question

I need to write a script that starts my program with different arguments. I start my program with:

./MyProgram.exe Data/data1.txt [Logs/data1_Log.txt].

Here is the pseudocode for what I want to do:

for each filename in /Data do
  for int i = 0, i = 3, i++
    ./MyProgram.exe Data/filename.txt Logs/filename_Log{i}.txt
  end for
end for

How can I create the second argument from the first one, so it looks like dataABCD_Log1.txt and start my program?

Answer 1

A couple of notes first: when you use Data/data1.txt as an argument, should it really be /Data/data1.txt (with a leading slash)? Also, should the outer loop scan only for .txt files, or all files in /Data? Here's an answer, assuming /Data/data1.txt and .txt files only:

#!/bin/bash
for filename in /Data/*.txt; do
    for ((i=0; i<=3; i++)); do
        ./MyProgram.exe "$filename" "Logs/$(basename "$filename" .txt)_Log$i.txt"
    done
done

Notes:

• /Data/*.txt expands to the paths of the text files in /Data (including the /Data/ part)
• $( ... ) runs a shell command and inserts its output at that point in the command line
• basename somepath .txt outputs the base part of somepath, with .txt removed from the end (e.g. /Data/file.txt -> file)

If you needed to run MyProgram with Data/file.txt instead of /Data/file.txt, use "${filename#/}" to remove the leading slash. On the other hand, if it's really Data not /Data you want to scan, just use for filename in Data/*.txt.

Answer 2

Whenever you iterate over files by globbing, it's good practice to avoid the corner case where the glob does not match (which makes the loop variable expand to the (un-matching) glob pattern string itself).

For example:

for filename in Data/*.txt; do
    [ -e "$filename" ] || continue
    # ... rest of the loop body
done

Reference: Bash Pitfalls

Answer 3

for file in Data/*.txt
do
    for ((i = 0; i < 3; i++))
    do
        name=${file##*/}
        base=${name%.txt}
        ./MyProgram.exe "$file" Logs/"${base}_Log$i.txt"
    done
done

The name=${file##*/} substitution (shell parameter expansion) removes the leading pathname up to the last /.

The base=${name%.txt} substitution removes the trailing .txt. It's a bit trickier if the extensions can vary.

Answer 4

You can use find's null-separated output option with read to iterate over directory structures safely.

#!/bin/bash
find . -type f -print0 | while IFS= read -r -d $'\0' file;
  do echo "$file" ;
done

So for your case,

#!/bin/bash
find . -maxdepth 1 -type f  -print0 | while IFS= read -r -d $'\0' file; do
  for ((i=0; i<=3; i++)); do
    ./MyProgram.exe "$file" 'Logs/'"`basename "$file"`""$i"'.txt'
  done
done

Additionally,

#!/bin/bash
while IFS= read -r -d $'\0' file; do
  for ((i=0; i<=3; i++)); do
    ./MyProgram.exe "$file" 'Logs/'"`basename "$file"`""$i"'.txt'
  done
done < <(find . -maxdepth 1 -type f  -print0)

will run the while loop in the current scope of the script (process) and allows the output of find to be used in setting variables, if needed.

Answer 5

Run a command on each file:

Do something (echo) with all .txt files,

for f in *.txt; do echo ${f}; done;

Answer 6

It looks like you're trying to execute a Windows file (.exe). Surely you ought to be using PowerShell. Anyway, on a Linux Bash shell a simple one-liner will suffice.

[/home/$] for filename in /Data/*.txt; do for i in {0..3}; do ./MyProgam.exe  Data/filenameLogs/$filename_log$i.txt; done done

Or in a Bash script:

#!/bin/bash

for filename in /Data/*.txt;
   do
     for i in {0..3};
       do ./MyProgam.exe Data/filename.txt Logs/$filename_log$i.txt;
     done
 done