How to pass a variable with a regular expression to egrep in bash

Question

I have variables where I store regular expressions and than I want to find files in local folder that match that regex

Basically, I have regex for filename and for its extensions which I want to concatenate and then use them in ereg to search for files

So, here is an example of how it looks like:

declare -r CPMOVE_REGEX="^cpmove-[a-z][a-z0-9]{0,16}"
declare -r TAR_REGEX=".tar$"

and then somewhere in the scriptI do the following.

for b in $(ls -1 | egrep "$CPMOVE_REGEX$TAR_REGEX");
do
  backups+=($b);
done

let's assume backups variable is declared somewhere before.
I understand that $CPMOVE_REGEX$TAR_REGEX is wrong, and I would like to know how to do it a right way.

Bash has `=~`, https://stackoverflow.com/a/35924143/1671066. — Micha Wiedenmann, Sep 01 '19 at 13:47
Possible duplicate of [How do I use a regex in a shell script?](https://stackoverflow.com/questions/35919103/how-do-i-use-a-regex-in-a-shell-script) — dmitriy ivanich, Sep 01 '19 at 14:20

score 0 · Answer 1 · answered Sep 01 '19 at 14:16

0

One of the ways to make it work is to put your vars in to curly brackets: "${CPMOVE_REGEX}${TAR_REGEX}"

Actually, I found it before posting this message, but I named my test files incorrectly, so they didn't match the regexp. So, that was the issue.

Thanks to Micha Wiedenmann who provided the link to the answer

answered Sep 01 '19 at 14:16

dmitriy ivanich

1
1

1

Curly braces shouldn't matter in this case. They don't hurt, but they're only needed when you need explicit control over where the variable reference ends. That is, when the variable name is followed by something that *could* be part of the variable name but isn't (like `"${var}text"` vs `"$vartext"`) or want to use a complex expansion (like `"${var/pattern/replace}"`). See [this question](https://stackoverflow.com/questions/8748831/when-do-we-need-curly-braces-around-shell-variables) for more info. – Gordon Davisson Sep 01 '19 at 17:58

Francesco Gasparetto · Answer 2 · 2019-09-02T08:20:00.410

0

Your regexp's problem is not in the extension part ($TAR_REGEX) but in the first part. Try to echo the whole string $CPMOVE_REGEX$TAR_REGEX and do some testing on the command line. When the regexp is ok, then put it in the script. There are no problems combining two variables into one string, and use it as egrep parameter, like in this working example:


root@localhost ~ $ declare -r P1='^[0-9]+\-.{3}'
root@localhost ~ $ ls | egrep "$P1"
20190315-GO2.sql
root@localhost ~ $ declare -r P2='.sql$'
root@localhost ~ $ ls | egrep "$P1$P2"
20190315-GO2.sql
root@localhost ~ $

edited Sep 02 '19 at 08:20

answered Sep 01 '19 at 14:21

Francesco Gasparetto

1,819
16
20

2

Please put double-quotes around those variable references! Without them, there's a possibility that the regex could be a gob-pattern-type match for one or more files, which would make a complete mess. – Gordon Davisson Sep 01 '19 at 18:00

KamilCuk · Answer 3 · 2019-09-02T08:52:35.643

how to do it a right way.

Probably something along:

declare -r CPMOVE_REGEX="cpmove-[a-z][a-z0-9]{0,16}"
declare -r TAR_REGEX=".tar"
IFS= readarray -d '' -t backups < <(find . -regextype posix-egrep -regex ".*/$CPMOVE_REGEX$TAR_REGEX" -print0)

Pre bashv4 doesn't has -d '' option with readarray. If you bash lower then v4, you need a while read loop:

while IFS= read -r tmp; do
    backups+=("$tmp")
done < <(find . -regextype posix-egrep -regex ".*/$CPMOVE_REGEX$TAR_REGEX" -print0)

But because your filenames have no whitespaces in them, which you filter with the regex, then if you are interested in filenames only without the directory paths, probably you could go with reading a newline separated list. That's only if you are sure you will never have any newlines in filenames.

IFS=$'\n' read -r -a backups < <(find . -regextype posix-egrep -regex ".*/$CPMOVE_REGEX$TAR_REGEX" -print "%f\n")

or

IFS=$'\n' backups=($(find . -regextype posix-egrep -regex ".*/$CPMOVE_REGEX$TAR_REGEX" -print "%f\n"))

Notes:

Do not parse ls output. It is not a tool to use in scripts. ls is for pretty printing in your console. Use find. Why you shouldn't parse the output of ls(1). Forget ls in scripts.
Doing for i in $( .. ) is error-prone, the $(...) undergoes word splitting. Although I think you are safe in your script, as you can do grep -x -E "<expression without whitespaces>", it's still conceptually better to use a while read loop. How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
egrep is deprecated, see the man page. Use grep -E instead.
Quote your variables. Don't backups+=($b) do backups+=("$b"). You will save yourself a lot of troubles later when you just quote all $ expansions. bashfaq quotes

How to pass a variable with a regular expression to egrep in bash

3 Answers3