Replace (literal) strings in a bash variable
Given bash variables containing:
- the original data,
- text to be replaced, and
- the replacement text
literal string replacement can be done with:
modified=${original/"$text"/"$replacement"}
The double-quotes are required.
See Shell Parameter Expansion for the nitty-gritty.
The complicated part is getting everything into the variables without any escaping.
Loading text into a bash variable without escaping
To avoid any quoting, it must be possible to choose a delimiter that cannot appear in the string. This is not possible in general but escaping can be minimised.
Single Quotes
If single-quotes (') are the delimiter, then any character except single quote may appear in the string:
var='this is a variable
containing lots of stuff !@#$%^&*()-=_+[]{};\:"|,./<>?
as well as trailing (and other) newlines:
'
Unfortunately single-quotes must still be escaped and included separately. For example:
var='anything except single quote ('\'')'
Here-docs
Another method is to read into the variable from a here-doc. In this case, the delimiter sequence cannot appear (EOD below). Be sure to use quotes around the delimiter, otherwise the input is treated like a double-quoted string and may have expansions performed.
var=$(cat <<'EOD'
this is a variable
containing lots of stuff !@#$%^&*()-=_+[]{};\:"|,./<>?
including single-quotes: '''
The delimiter (`EOD`) cannot appear alone on a line.
EOD
)
However note that with this method any trailing newlines are not stored. To include them mapfile can be used if available (bash 4+) or a sentinel may be appended and stripped:
mapfile -d '' var <<'EOD'
Anything allowed (except `EOD` alone on a line).
Trailing newlines are retained:
EOD
tmpvar=$(cat <<'EOD'
String containing anything (except `EOD` alone on a line).
Trailing newlines can't exist because of the sentinel:
sentinel
EOD
)
var=${tmpvar%sentinel}
Actual files
If content for a variable is stored in an actual file, the delimiter can be the null character (which cannot be stored in a bash variable and so is guaranteed not to appear). See below.
Loading data from file
If the original data is stored in a file rather than in a variable, it must first be read in. On versions of bash that have mapfile, this can be safely done with:
mapfile -d '' var <"$path_to_file"
Otherwise the sentinel trick can be used:
tmpvar=$(cat "$path_to_file"; echo sentinel)
var=${tmpvar%sentinel}
If trailing newlines don't exist (or should be elided), bash provides a slightly faster alternative to cat to read from a file:
var=$(< "$path_to_file")
Note that these methods can also be used to load the text and replacement variables with arbitrary text stored in files.
Writing data out to file
Saving the modified data can be done with printf:
printf '%s' "$modified" >"$path_to_file"
printf is more reliable than echo because different implementations of echo behave differently and the string printed may get mangled. For example, consider modified='-n'; echo "$modified" or if modified should not end with a newline for some reason.
See: https://unix.stackexchange.com/q/65803/333919
Putting it all together
mapfile -d '' text <<'EOD'
...anything (except EOD alone on a line)...
...(or use sentinel trick (without a sentinel!) if text does not
... end with a newline; or a single-quoted string if that is simpler)
EOD
mapfile -d '' replacement <<'EOD'
...anything else (except EOD alone on a line)..
...(again, use sentinel trick if newline handling not required)
EOD
mapfile -d '' original <"$path_to_file"
printf '%s' "${original/"$text"/"$replacement"}" >"$path_to_file"
