Sed inline dynamic substitution expression throws: unmatched '|'

Question

I have a minified js file. Contains special characters like pipes, dashes, slashes, etc.

Also, I've another HTML file that contains a comment inside:

<!--#MY_SNIPPET#-->

I'm trying to use sed to inject my js file and replace it with the comment above.

Example of my.js

<script>whatever|@</script><script>-my</script>

I'm currently doing:

VARIABLE=$(cat my.js)
sed -i "s|<!--#MY_SNIPPET#-->|${VARIABLE}|" index.html

But I'm getting sed: unmatched '|'

I assume that sed is trying to evaluate the string and that's why is failing but I'm wondering if there's an easy way to make it possible just using sed because I'd like to avoid using Perl or anything else.

Doesn't matter the delimiter, it always throws the same error.

your sample `my.js` contains `|` which is same as delimiter for `s`.. use something else - may be `;`? see also https://stackoverflow.com/questions/29613304/is-it-possible-to-escape-regex-metacharacters-reliably-with-sed — Sundeep, May 05 '18 at 12:58
I would also suggest to use `r` command and avoid all these surprises in first place.. for GNU sed, see this for example https://stackoverflow.com/a/39413525 — Sundeep, May 05 '18 at 12:59
@Sundeep doesn't matter which delimiter I use, it throws always `sed: unmatched 'DELIMITER'` — Idir Ouhab Meskine, May 05 '18 at 13:10
you're definitely better off using `r` command... another issue could be `!` inside double quotes.. so try something like `sed 's;;'"${VARIABLE}"';'` — Sundeep, May 05 '18 at 13:13
**Break the problem in two.** Try `sed "//d" index.html`, and tinker with that until you get it working perfectly, before you try to insert the script. Meanwhile, get `sed "5r my.js" index.html` working perfectly, before you attempt to match the "snippet" line. — Beta, May 05 '18 at 15:14

LMC · Answer 1 · 2018-05-05T14:39:05.927

Building a dynamic sed expression can be difficult if not impossible in some cases. Yours has 2 problems: The exclamation mark ! inside (or outside) double quotes has bash special purposes, e.g. try these on a console

!! This one repeats the last executed command (be careful! :D )
echo "!$" echo the last argument of the previous command
!5087 execute command 5087 in your bash history.

The other big problem is that your variable may contain almost any character so you can't be sure that the sed expression delimiter is no present there, for your particular example, this works

sed -ri "s~<[!]--#MY_SNIPPET#-->~${var}~" test.html

PS: delimiter must be a single byte character :(

sed -ri "sЖ<[!]--#MY_SNIPPET#-->Ж${var}Ж" test.html

sed: -e expression #1, char 2: delimiter character is not a single-byte character

Another twisted option could be to replace the chosen delimiter in VARIABLE and then restore it

sed -ri "s@<[!]--#MY_SNIPPET#-->@$(echo "${var}" | tr '@' 'ж')@" test.html
cat test.html | tr 'ж' '@' > test2.html
mv test2.html test.html

Ed Morton · Answer 2 · 2018-05-05T16:46:59.387

Never enclose a whole script in double quotes as you're then exposing the whole script to the shell for interpretation and that can have unforeseen consequences like you're experiencing right now. Only open up the script to the shell if/when you need the shell to do a specific task, e.g. expand a variable in your case. Enclose scripts in single quotes, not double.

So, don't write:

sed -i "s|<!--#MY_SNIPPET#-->|${VARIABLE}|" index.html

write this instead:

sed -i 's|<!--#MY_SNIPPET#-->|'"${VARIABLE}"'|' index.html

Now your next problem is that VARIABLE contains the delimiter you're using in your script, |. To fix that, just change the delimiter in the script, e.g. from | to ~:

sed -i 's~<!--#MY_SNIPPET#-->~'"${VARIABLE}"'~' file

That will "work" as long as VARIABLE doesn't contain any ~, &, or \<number> chars/strings. For a more robust solution you need to do what it shows at Is it possible to escape regex metacharacters reliably with sed or switch to awk since awk can operator on literal strings while sed cannot.

score 0 · Answer 3 · answered May 05 '18 at 20:07

No matter what delimiter you use for the s/// command, there's a risk your variable might contain that character. The best practice would be to escape any such character in the variable:

$ var='<script>whatever|@</script><script>-my</script>'
$ echo "${var//|/\\|}"
<script>whatever\|@</script><script>-my</script>
# ..............^^

then

sed -i "s|<!--#MY_SNIPPET#-->|${var//|/\\|}|" index.html

Sed inline dynamic substitution expression throws: unmatched '|'

3 Answers3