0

I am currently trying to batch convert a bunch of html files to txt files using html2txt inside a python virtual environment. My script as-is currently is the following:

#!/usr/bin/bash

NAME=""
source ~/pythonenvs/env_tomboyconversion/bin/activate

for i in $(ls *.html); do
    NAME=$(basename -s .html $i)
    html2text $i > "$NAME.txt"
done

deactivate

Whenever I run the script, I end up with with multiple empty text files where it's obvious the original filename was split on spaces. For example, if I have a file original file.html, it will result in original.txt and file.txt. The problem I'm having is I can't figure out where to apply quotes in order to keep filenames from being split on the spaces. I've tried the above, as well as putting the $i, around the $(basename -s .html $i) expression, and various combinations thereof, but the filenames keep getting split. How can I get the filenames to stop splitting on the spaces?

alyms108
  • 3

1 Answers1

0

This is the improved loop:

shopt -s nullglob
for i in ./*.html; do
    name="$(basename -s .html "$i")"
    html2text "$i" > "$name.txt"
done

Improvements:

  • Bash pitfall number 1: for i in $(ls *.html); do is flawed. One of the problems:

    If a filename contains whitespace, it undergoes WordSplitting.

    But there are more. You cannot fix it by quoting. In your case there is no reason to call ls at all. Using *.html directly is the right way.

  • Proper double-quoting. Use double-quotes around variable substitutions and command substitutions. In this particular line

    name="$(basename -s .html "$i")"

    the quotes are nested (see this, quirk 2). Here the outer ones are optional because variable assignment in a form

    foo=$(command_that_can_output_spaces_or_such)

    will work anyway, but this only works when assigning in this simple way. In general you do want to double-quote $(…). E.g you may not use the name variable at all:

    html2text "$i" > "$(basename -s .html "$i").txt"

    and in this case you shouldn't omit any quotes.

  • NAME changed to name. See this answer:

    Environment variables or shell variables that are introduced by the operating system or shell startup scripts etc. are usually all in CAPITALS.

    To prevent your own variables from conflicting with these variables, it is a good practice to use lower case.

  • Each $i begins with ./. This prevents other tools from interpreting it as an option (if the variable value happens to begin with -). There is another way to deal with it: --. I think basename understands -- but I'm not sure about html2text, therefore the trick with ./

  • shopt -s nullglob is useful when there is no match for ./*.html. Normally in such case the pattern would not expand and the loop would run once with $i being the literal string ./*.html. This is not what you want. shopt -s nullglob allows the pattern to expand to nothing, so the loop will not run at all if there is no match. This is bashism, it will work in Bash but other shells may/will not understand it.

You don't need basename to remove the extension, the shell can do this on its own:

shopt -s nullglob
for i in ./*.html; do
    html2text "$i" > "${i%.html}.txt"
done

This approach avoids spawning an extra process (basename).

Kamil Maciorowski
  • 81,893