How to loop multiple extensions with upper and lower case letters using for command

Question

I am trying to loop a set of jpg images in a folder. The problem is some of the images have the extension in all lower case letters (.jpg) and others have all capital letters (.JPG).

I am trying to use variable modifiers to change the extensions as the loop progresses.

I figured out how to phrase the for var in LIST part but I can't get the modifiers to replace the extensions correctly.

Essentially I need the jpg files to convert to files with .mpc and .cache extensions.

I end up with files that have names like FILE.jpg.mpc or FILE.JPG.jpg when I need the file to be FILE.mpc and FILE.cache

Here is my function that works great if you only use for i in *.jpg but when you add for i in *.{jpg,JPG} everything falls apart.

Again here is what I have so far.

imow()
{

    clear

    local i DIMENSIONS RANDOM

    # find all jpg files and create temporary cache files from them
    for i in *.{jpg,JPG}
    do
        # create random direcotories in case you are running this function more than once at the same time. it prevents cross-over.
        RANDOM="$(mktemp --directory)"
        "${RANDOM}" 2>/dev/null
        echo -e "\\nCreating two temporary cache files: ${RANDOM}/${i%%.jpg,JPG}.mpc + ${RANDOM}/${i%%.{jpg,JPG}}.cache\\n"
        DIMENSIONS="$(identify -format '%wx%h' "${i}")"
        convert "${i}" -monitor -filter 'Triangle' -define filter:support='2' -thumbnail "${DIMENSIONS}" -strip \
        -unsharp '0.25x0.08+8.3+0.045' -dither None -posterize '136' -quality '82' -define jpeg:fancy-upsampling='off' \
        -define png:compression-filter='5' -define png:compression-level='9' -define png:compression-strategy='1' \
        -define png:exclude-chunk='all' -auto-level -enhance -interlace 'none' -colorspace 'sRGB' "${RANDOM}/${i%%.{jpg,JPG}}.mpc"
        clear
        for i in "${RANDOM}"/*.mpc
        do
            if [ -f "${i}" ]; then
                echo -e "\\nOverwriting original file with optimized self: ${i} >> ${i%%.mpc}.jpg\\n"
                convert "${i}" -monitor "${i%%.mpc}.jpg"
                if [ -f "${i%%.mpc}.jpg" ]; then
                    mv "${i%%.mpc}.jpg" "${PWD}"
                    rm -fr "${RANDOM}"
                    clear
                else
                    clear
                    echo 'Error: Unable to find the optimized image and therefore can'\''t overwrite the original.'
                    echo
                    exit 1
                fi
            fi
        done
    done
}

Like I mentioned if you run it with just the .jpg extension it works great. here is a working example.

imow()
{

    clear

    local i DIMENSIONS RANDOM

    # find all jpg files and create temporary cache files from them
    for i in *.jpg
    do
        # create random directories in case you are running this function more than once at the same time. it prevents cross-over.
        RANDOM="$(mktemp --directory)"
        "${RANDOM}" 2>/dev/null
        echo -e "\\nCreating two temporary cache files: ${RANDOM}/${i%%.jpg}.mpc + ${RANDOM}/${i%%.jpg}.cache\\n"
        DIMENSIONS="$(identify -format '%wx%h' "${i}")"
        convert "${i}" -monitor -filter 'Triangle' -define filter:support='2' -thumbnail "${DIMENSIONS}" -strip \
        -unsharp '0.25x0.08+8.3+0.045' -dither None -posterize '136' -quality '82' -define jpeg:fancy-upsampling='off' \
        -define png:compression-filter='5' -define png:compression-level='9' -define png:compression-strategy='1' \
        -define png:exclude-chunk='all' -auto-level -enhance -interlace 'none' -colorspace 'sRGB' "${RANDOM}/${i%%.jpg}.mpc"
        clear
        for i in "${RANDOM}"/*.mpc
        do
            if [ -f "${i}" ]; then
                echo -e "\\nOverwriting original file with optimized self: ${i} >> ${i%%.mpc}.jpg\\n"
                convert "${i}" -monitor "${i%%.mpc}.jpg"
                if [ -f "${i%%.mpc}.jpg" ]; then
                    mv "${i%%.mpc}.jpg" "${PWD}"
                    rm -fr "${RANDOM}"
                    clear
                else
                    clear
                    echo 'Error: Unable to find the optimized image and therefore can'\''t overwrite the original.'
                    echo
                    exit 1
                fi
            fi
        done
    done

Answer 1

${i%%.jpg,JPG} and ${i%%.{jpg,JPG}} don't do what you want. The first tries to remove the exact suffix .jpg,JPG, and the second tries to remove the exact suffix .{jpg,JPG}. Since the suffix is guaranteed to be either .jpg or .JPG at that point in the code, ${i%.*} is a safe alternative.

Answer 2

As I read it, your function already overwrites original .jpg files, but hardcodes a lowercase .jpg which leaves the old .JPG files lying around. I'd just formalize the conversion to lowercase unless there's some specific reason you shouldn't/can't.

Get rid of those broken uppercase variables, and your inner loop using the same loop var as the outer loop, etc. I moved some stuff around, but don't have imagemagick installed so this is a freehand edit; I trust someone will point out any glaring logic errors as I can't really test it the way I'd like.

I explicitly test the imagemagick return code to simplify a bit, and added a test to make sure the mv succeeded. If the original file's extension is not all lowercase, it now should be removed once the optimized file is staged.

I also removed the clear commands that were erasing output you had just explicitly logged in favor of sending convert spam to logs that get wiped unless there was a problem.

imow() {
  local orig mpc new dim rc tdir boilerplate
  tdir="$(mktemp --directory)" 
  boilerplate=( -monitor -filter 'Triangle' -define filter:support='2' 
    -strip -unsharp '0.25x0.08+8.3+0.045' -dither None -posterize '136'
    -quality '82' -define jpeg:fancy-upsampling='off' 
    -define png:compression-filter='5' -define png:compression-level='9'
    -define png:compression-strategy='1' -define png:exclude-chunk='all'
    -auto-level -enhance -interlace 'none' -colorspace 'sRGB' )
  for orig in *.[Jj][Pp][Gg]
  do dim="$(identify -format '%wx%h' "$orig")"
     if convert "$orig" "${boilerplate[@]}" -thumbnail "$dim" "${tdir}/${orig%.???}.mpc" > "$tdir/convert-mpc.out"
     then for mpc in "${tdir}"/*.mpc
          do new="${mpc%.mpc}.jpg"
             if convert "$mpc" -monitor "$new" > "$tdir/convert-jpg.out"
             then echo "Replacing $orig with optimized version $new"
                  if mv "$new" "$PWD"
                  then [[ "$orig" == "$new" ]] || rm -f "$orig"
                       rm -fr "${tdir:?safety check}/*.*" # can't expand to rm -fr /*
                  else rc=$?;
                       echo "Cannot mv '$new' to '$PWD': error $rc; aborting" >&2
                       exit $rc
                  fi
        else rc=$?
                  echo "convert error $rc on $mpc; c.f. $tdir - aborting." >&2
                  exit $rc
             fi
          done
     else rc=$?
          echo "convert error $rc on $orig; c.f. $tdir - aborting." >&2
          exit $rc
     fi
  done
  rm -fr "$tdir"
}

*.[Jj][Pp][Gg] with match any combination of upper/lower on the extension.

Make sure you test and modify to suit your needs.
Good luck.

addendum

I have discovered that changing the line if convert "${mpc}" -monitor "${new}" > "${tdir}/convert-jpg.out" to if convert "${mpc}" -monitor "${new}" > "${tdir}/convert-jpg.out" &> /dev/null GREATLY speeds up the parsing of each loop. Otherwise cheers.

Please c.f. the GNU bash guide on Redirections

Observe:

$: date >foo
$: cat foo
Fri Feb  3 12:33:13 CST 2023
$:

whereas:

$: date >foo &> /dev/null
$: cat foo
$:

in the second, date's output is redirected to foo, which opens (and creates or truncates) the file, but then &> redirects both stdout and stderr to /dev/null, overriding the first redirection to a logfile, so foo is empty after it runs.

Maybe it was "significantly faster" because it failed?
How would you know?

IMHO, never throw away your logs, especially stderr, even if you are testing your return. When it eventually does fail, you have nothing to debug. I usually clean up the logs after a success, but on a fail I usually cat them, then and abort.

Worst case, even if you do throw them away, remove the redundancy.
Assuming all use

if convert "${mpc}" -monitor "${new}" 

followed by some output control or other, rather than

 > "${tdir}/convert-jpg.out" &> /dev/null # log will always be empty

If you just do not want the stdout at all, try one of these -

Leaving stderr defaulting to the console,

> /dev/null                              # stdout to the bitbucket
>&-                                      # just *close* stdout

redirecting stderr to a log,

2> "${tdir}/convert-jpg.out" > /dev/null # stdout to the bitbucket
2> "${tdir}/convert-jpg.out" >&-         # stdout closed

if stdout has any value -

&> "${tdir}/convert-jpg.out"             # both to a specified log

or worst case, if you just insist,

&>/dev/null                              # BOTH to the bitbucket
&>-                                      # close both

Answer 3

Does this work for you?

shopt -s nullglob
cd
cd desktop/tests
for i in *.jpg *.JPG; do
echo $i
done