sed substitution with Bash variables

Question

I am trying to change the values in a text file using sed in a Bash script with the line,

sed 's/draw($prev_number;n_)/draw($number;n_)/g' file.txt > tmp

This will be in a for loop. Why is it not working?

Answer 1

Variables inside ' don't get substituted in Bash. To get string substitution (or interpolation, if you're familiar with Perl) you would need to change it to use double quotes " instead of the single quotes:

# Enclose the entire expression in double quotes
$ sed "s/draw($prev_number;n_)/draw($number;n_)/g" file.txt > tmp

# Or, concatenate strings with only variables inside double quotes
# This would restrict expansion to the relevant portion
# and prevent accidental expansion for !, backticks, etc.
$ sed 's/draw('"$prev_number"';n_)/draw('"$number"';n_)/g' file.txt > tmp

# A variable cannot contain arbitrary characters
# See link in the further reading section for details
$ a='foo
bar'
$ echo 'baz' | sed 's/baz/'"$a"'/g'
sed: -e expression #1, char 9: unterminated `s' command

Further Reading:

• Difference between single and double quotes in Bash
• Is it possible to escape regex metacharacters reliably with sed
• Using different delimiters for sed substitute command
• Unless you need it in a different file you can use the -i flag to change the file in place

Answer 2

Variables within single quotes are not expanded, but within double quotes they are. Use double quotes in this case.

sed "s/draw($prev_number;n_)/draw($number;n_)/g" file.txt > tmp

You could also make it work with eval, but don’t do that!!

Answer 3

This may help:

sed "s/draw($prev_number;n_)/draw($number;n_)/g" 

Answer 4

You can use variables like below. Like here, I wanted to replace hostname i.e., a system variable in the file. I am looking for string look.me and replacing that whole line with look.me=<system_name>

sed -i "s/.*look.me.*/look.me=`hostname`/"

You can also store your system value in another variable and can use that variable for substitution.

host_var=`hostname`

sed -i "s/.*look.me.*/look.me=$host_var/"

---

Input file:

look.me=demonic

Output of file (assuming my system name is prod-cfm-frontend-1-usa-central-1):

look.me=prod-cfm-frontend-1-usa-central-1

Answer 5

I needed to input github tags from my release within github actions. So that on release it will automatically package up and push code to artifactory.

Here is how I did it. :)

  - name: Invoke build
    run: |
      # Gets the Tag number from the release
      TAGNUMBER=$(echo $GITHUB_REF | cut -d / -f 3)
      
      # Setups a string to be used by sed
      FINDANDREPLACE='s/${GITHUBACTIONSTAG}/'$(echo $TAGNUMBER)/
      
      # Updates the setup.cfg file within version number
      sed -i $FINDANDREPLACE setup.cfg
      
      # Installs prerequisites and pushes 
      pip install -r requirements-dev.txt
      invoke build

Retrospectively I wish I did this in python with tests. However it was fun todo some bash.

Answer 6

Another variant, using printf:

SED_EXPR="$(printf -- 's/draw(%s;n_)/draw(%s;n_)/g' $prev_number $number)"

sed "${SED_EXPR}" file.txt

or in one line:

sed "$(printf -- 's/draw(%s;n_)/draw(%s;n_)/g' $prev_number $number)" file.txt

Using printf to build the replacement expression should be safe against all kinds of weird things, which is why I like this variant.