Looping through files of specified extensions in bash

Question

I am trying to loop through files of a list of specified extensions with a bash script. I tried the solution given at Matching files with various extensions using for loop but it does not work as expected. The solution given was:

for file in "${arg}"/*.{txt,h,py}; do

Here is my version of it:

for f in "${arg}"/*.{epub,mobi,chm,rtf,lit,djvu}
    do
        echo "$f"
    done

When I run this in a directory with an epub file in it, I get:

/*.epub
/*.mobi
/*.chm
/*.rtf
/*.lit
/*.djvu

So I tried changing the for statement:

for f in "${arg}"*.{epub,mobi,chm,rtf,lit,djvu}

Then I got:

089281098X.epub
*.mobi
*.chm
*.rtf
*.lit
*.djvu

I also get the same result with:

for f in *.{epub,mobi,chm,rtf,lit,djvu}

So it seems that the "${arg}" argument is unnecessary.

Although either of these statements finds files of the specified extensions and can pass them to a program, I get read errors from the unresolved *. filenames.

I am running this on OS X Mountain Lion. I was aware that the default bash shell was outdated so I upgraded it from 3.2.48 to 4.2.45 using homebrew to see if this was the problem. That didn't help so I am wondering why I am getting these unexpected results. Is the given solution wrong or is the OS X bash shell somehow different from the *NIX version? Is there perhaps an alternate way to accomplish the same thing that might work better in the OS X bash shell?

Answer 1

This may be a BASH 4.2ism. It does not work in my BASH which is still 3.2. However, if you shopt -s extglob, you can use *(...) instead:

shopt -s extglob
for file in *.*(epub|mobi|chm|rtf|lit|djvu)
do
    ...
done

---

@David W.: shopt -s extglob for f in .(epub|mobi|chm|rtf|lit|djvu) results in: 089281098X.epub @kojiro: arg=. shopt -s nullglob for f in "${arg}"/.{epub,mobi,chm,rtf,lit,djvu} results in: ./089281098X.epub shopt -s nullglob for f in "${arg}".{epub,mobi,chm,rtf,lit,djvu} results in: 089281098X.epub So all of these variations work but I don't understand why. Can either of you explain what is going on with each variation and what ${arg} is doing? I would really like to understand this so I can increase my knowledge. Thanks for the help.

In mine:

for f in *.*(epub|mobi|chm|rtf|lit|djvu)

I didn't include ${arg} which expands to the value of $arg. The *(...) matches the pattern found in the parentheses which is one of any of the series of extensions. Thus, it matches *.epub.

Kojiro's:

arg=. 
shopt -s nullglob 
for f in "${arg}"/*.{epub,mobi,chm,rtf,lit,djvu}

Is including $arg and the slash in his matching. Thus, koriro's start with ./ because that's what they are asking for.

It's like the difference between:

echo *

and

echo ./*

By the way, you could do this with the other expressions too:

echo *.*(epub|mobi|chm|rtf|lit|djvu)

The shell is doing all of the expansion for you. It's really has nothing to do with the for statement itself.

Answer 2

A glob has to expand to an existing, found name, or it is left alone with the asterisk intact. If you have an empty directory, *.foo will expand to *.foo. (Unless you use the nullglob Bash extension.)

The problem with your code is that you start with an arg, $arg, which is apparently empty or undefined. So your glob, ${arg}/*.epub expands to /*.epub because there are no files ending in ".epub" in the root directory. It's never looking in the current directory. For it to do that, you'd need to set arg=. first.

In your second example, the ${arg}*.epub does expand because $arg is empty, but the other files don't exist, so they continue not to expand as globs. As I hinted at before, one easy workaround would be to activate nullglob with shopt -s nullglob. This is bash-specific, but will cause *.foo to expand to an empty string if there is no matching file. For a strict POSIX solution, you would have to filter out unexpanded globs using [ -f "$f" ]. (Then again, if you wanted POSIX, you couldn't use brace expansion either.)

Answer 3

To summarize, the best solutions are to use (most intuitive and elegant):

shopt -s extglob
for f in *.*(epub|mobi|chm|rtf|lit|djvu)

or, in keeping with the original solution given in the referenced thread (which was wrong as stated):

shopt -s nullglob
for f in "${arg}"*.{epub,mobi,chm,rtf,lit,djvu}

Answer 4

This should do it:

for file in $(find ./ -name '*.epub' -o -name '*.mobi' -o -name '*.chm' -o -name '*.rtf' -o -name '*.lit' -o -name '*.djvu'); do
  echo $file
done