sed: replacing a string with a multi-line file's contents

Question

I'm trying to replace a line with a string, but receiving either unknown option to s' or unterminated s' command errors from sed. Using sigils other than / (tried both @ and #) has no effect.

line='<script type="text/javascript" src="<?php echo $jquery_path ?>jQuery.js"></script>'
file_content=$(sed 's:/:\\/:g' jquery.js) # escape /s in replacement content

sed -i -e "s@$line@$file_content@" ./index.php

jquery.js is minified source, and should not contain any newlines.

...in your outer `sed` operation, that is. The inner one may still have problems, but since we don't have your `jquery.js`, we can't reproduce its output to tell what the outer one will actually contain. — Charles Duffy, Aug 04 '15 at 16:14
Thus, as currently asked, this question doesn't contain enough information to be answerable. I'd suggest running with `set -x` first to show the commands being run post-expansion. — Charles Duffy, Aug 04 '15 at 16:15
Anyhow, let's back up and go over this: You're running a second `sed` operation, inside your first one, and substituting its output into the first `sed`. We don't have the files you're using as input to that second `sed` operation, so we don't know what it's returning. Thus, we don't know what you're **actually** passing to your outer `sed` instance, so we can't possibly help you unless you edit your question to add that information. More clear now? — Charles Duffy, Aug 04 '15 at 16:19
Also, see http://stackoverflow.com/help/mcve -- code incorporated in questions needs to be **complete** enough for someone else to reproduce the problem. You're depending on having a `../my_files/jquery.js` with unspecified contents; that makes your question incomplete, because nobody not having that file can reproduce your problem. — Charles Duffy, Aug 04 '15 at 16:19
I'm guessing, by the way, that what's returned from that operation contains newlines; that would explain what we're seeing here, since newlines terminate `sed` commands. — Charles Duffy, Aug 04 '15 at 16:20
I am using jquery (minified version). This plugin is kind of popular. I just thought that it would be useless to tell about this file cause its name speeks for itself. About terminating new lines, How can I replace a string with a file content so that it worked fine? not particulary sed. PS this file is minified to one line. And abbout where I got this method from,, reading these kinds of posts: http://unix.stackexchange.com/questions/141387/sed-replace-string-with-file-contents — jackson, Aug 04 '15 at 16:25
I'd suggest using gsub_literal from http://mywiki.wooledge.org/BashFAQ/021 — Charles Duffy, Aug 04 '15 at 16:26
The filename doesn't "speak for itself" in terms of whether the contents are minified, which is important in cases (such as here) where whitespace matters. If it were jquery.min.js, then yes, it'd be enough of a hint. — Charles Duffy, Aug 04 '15 at 16:30
BTW -- the official jquery.min.js builds do contain a newline between the comment with licensing info and the source. Did you remove that in your copy? — Charles Duffy, Aug 04 '15 at 16:41

Charles Duffy · Accepted Answer · 2015-08-04T16:47:16.887

In terms of what went wrong:

jquery.min.js contains @ and # characters, so neither of these is safe as an unescaped sigil.
Your sed operation on jquery.min.js is escaping /s, even if you're using @s in the outer sed instance.

To fix this, you might change
```
file_content=$(sed 's:/:\\/:g' jquery.js)
```
...to...
```
file_content=$(sed 's:@:\\@:g' jquery.js)
```

Also, if you didn't remove the first line with a comment with licensing information, the replacement file isn't actually only one line. Putting an unescaped newline in sed command terminates it.

The easy answer is not to use sed at all. Consider, for instance, gsub_literal, as defined in BashFAQ #21:

gsub_literal() {
  # STR cannot be empty
  [[ $1 ]] || return

  # string manip needed to escape '\'s, so awk doesn't expand '\n' and such
  awk -v str="${1//\\/\\\\}" -v rep="${2//\\/\\\\}" '
    # get the length of the search string
    BEGIN {
      len = length(str);
    }

    {
      # empty the output string
      out = "";

      # continue looping while the search string is in the line
      while (i = index($0, str)) {
        # append everything up to the search string, and the replacement string
        out = out substr($0, 1, i-1) rep;

        # remove everything up to and including the first instance of the
        # search string from the line
        $0 = substr($0, i + len);
      }

      # append whatever is left
      out = out $0;

      print out;
    }
  '
}

In your use case, that might look like the following:

gsub_literal \
  '<script type="text/javascript" src="<?php echo $jquery_path ?>jQuery.js' \
  "$(<../my_files/jquery.js)" \
  <index.php >index.php.out && mv index.php.out index.php

Isn't AWK for miner files (the ones that are not large in size)? — jackson, Aug 04 '15 at 16:31
Quite the opposite -- `awk` is much faster processing large streams than native bash is. By contrast, its startup time makes it wasteful to use for one-line inputs which could be handled natively in bash. — Charles Duffy, Aug 04 '15 at 16:31
Hey, Thanks a lot to you! It really works! I have just one quetion: what is index.php.out for? — jackson, Aug 04 '15 at 16:38
You can't safely set up a pipeline that reads and writes to the same location -- this is because `foo out.txt` opens both `in.txt` and `out.txt` *before* it starts `foo`, so `out.txt` is deleted before `foo` is run in this case. Similarly, if you tried to run `foo index.php`, you'd find `index.php` to be empty before `foo` could even read from it. — Charles Duffy, Aug 04 '15 at 16:41
Thank you very much for the detailed answer and for answering my question. If I had a bit more reputation, I wouldd vote you up. — jackson, Aug 04 '15 at 16:44

score 0 · Answer 2 · edited May 23 '17 at 11:58

You said yourself that you want to replace a string. sed CANNOT operate on strings, only on regexps (see Is it possible to escape regex metacharacters reliably with sed). awk on the other hand can operate on strings so just use awk:

awk -v old='<script type="text/javascript" src="<?php echo $jquery_path ?>jQuery.js"></script>' '
NR == FNR { new = new $0 ORS; next }
$0 == old { $0 = new }
{ print }
' jquery.js index.php

sed: replacing a string with a multi-line file's contents

2 Answers2