Loop through all the files with .txt extension in bash

Question

I am trying to loop over files in a folder and test for .txt extensions. But I get the following error: "awk: cannot open = (No such file or directory)

Here's my code:

!/bin/bash

files=$(ls);
for file in $files
  do
  # extension=$($file | awk -F . '{ print $NF }');
  if [ $file | awk -F . "{ print $NF }" = txt ]
    then
      echo $file;
    else
      echo "Not a .txt file";
  fi;
done;

Can't you just use globbing with `for i in *.txt; do ... done` ? — oliv, May 04 '18 at 10:53
@oliv, but first I have to list all of the files in the folder I am in? Right? — Faris Marouane, May 04 '18 at 11:01

Inian · Accepted Answer · 2018-05-04T12:17:54.957

The way you are doing this is wrong in many ways.

You should never parse output of ls. It does not handle the filename containing special characters intuitively See Why you shouldn't parse the output of ls(1)
Don't use variables to store multi-line data. The output of ls in a variable is expected to undergo word splitting. In your case files is being referenced as a plain variable, and without a delimiter set, you can't go through the multiple files stored.
Using awk is absolutely unnecessary here, the part $file | awk -F . "{ print $NF }" = txt is totally wrong, you are not passing the name the file to the pipe, just the variable $file, it should have been echo "$file"
The right interpreter she-bang should have been set as #!/bin/bash in your script if you were planning to run it as an executable, i.e. ./script.sh. The more recommended way would be to say #!/usr/bin/env bash to let the shell identify the default version of the bash installed.

As such your requirement could be simply reduced to

for file in *.txt; do
    [ -f "$file" ] || continue
    echo "$file"
done

This is a simple example using a glob pattern using *.txt which does pathname expansion on the all the files ending with the txt format. Before the loop is processed, the glob is expanded as the list of files i.e. assuming the folder has files as 1.txt, 2.txt and foo.txt, the loop is generated to

for file in 1.txt 2.txt foo.txt; do

Even in the presence of no files, i.e. when the glob matches empty (no text files found), the condition [ -f "$file" ] || continue would ensure the loop is exit gracefully by checking if the glob returned any valid file results or just an un-expanded string. The condition [ -f "$file" ] would fail for everything if except a valid file argument.

Or if you are targeting scripts for bourne again shell, enable glob options to remove non-matching globs, rather than preserving them

shopt -s nullglob
for file in *.txt; do
    echo "$file"
done

Another way using shell array to store the glob results and parse them over later to do a specific action on them. This way is useful when doing a list of files as an argument list to another command. Using a proper quoted expansion "${filesList[@]}" will preserve the spacing/tabs/newlines and other meta characters in filenames.

shopt -s nullglob
filesList=(*.txt)

for file in "${filesList[@]}"; do
    echo "$file"
done

Loop through all the files with .txt extension in bash

1 Answers1