2

I am porting a sh script that was apparently written using GNU implementation of sed to BSD implementation of sed. The exact line in the script with the original comment are:

# escape dot in file extension to grep it
ext="$(echo $ext | sed 's/\./\\./' -)"

I am able to reproduce a results with the following (obviously I am not exhausting all possibilities values for ext) :

ext=.h; ext="$(echo $ext | sed 's/\./\\./' -)"; echo [$ext]

Using GNU's implementation of sed the following is returned:

[\.h]

Using BSD's implementation of sed the following is returned:

sed: -: No such file or directory
[]

Executing ext=.h; ext="$(echo $ext | sed 's/\./\\./')"; echo [$ext] returns [\.h] for both implementation of sed.

I have looked at both GNU and BSD's sed's man page have not found anything about the trailing "-". Googling for sed with a "-" is not very fruitful either.

Is the "-" a typo? Is the "-" needed for some an unexpected value of $ext? Is the issue not with sed, but rather with sh?

Can someone direct me to what I should be looking at, or even better, explain what the purpose of the "-" is?

mmorris
  • 4,006
  • 3
  • 27
  • 29

2 Answers2

1

On my system, that syntax isn't documented in the man page, but it is in the 'info' page:

sed OPTIONS... [SCRIPT] [INPUTFILE...]

If you do not specify INPUTFILE, or if INPUTFILE is -',sed' filters the contents of the standard input.

Given that particular usage, I think you could leave off the '-' and it should still work.

Jim Lewis
  • 43,505
  • 7
  • 82
  • 96
0

You got your specific question answered BUT your script is all wrong. Take a look at this:

# escape dot in file extension to grep it
ext="$(echo $ext | sed 's/\./\\./')"

The main problems with that are:

  1. You're not quoting your variable ($ext) so it will go through file name expansion plus if it contains spaces will be passed to echo as multiple arguments instead of 1. Do this instead:

    ext="$(echo "$ext" | sed 's/\./\\./')"

  2. You're using an external command (sed) and a pipe to do something the shell can do trivially itself. Do this instead:

    ext="${ext/./\.}"

  3. Worst of all: You're escaping the RE meta-character (.) in your variable so you can pass it to grep to do an RE search on it as if it were a string - that doesn't make any sense and becomes intractable in the general case where your variable could contain any combination of RE metacharacters. Just do a string search instead of an RE search and you don't need to escape anything. Don't do either of the above substitution commands and then do either of these instead of grep "$ext" file:

    grep -F "$ext" file
fgrep "$ext" file
awk -v ext="$ext" 'index($0,ext)' file
Ed Morton
  • 188,023
  • 17
  • 78
  • 185
  • His script is fine. #1. Perhaps ext has no problematic characters (likely if it's an extension). #2. No! That's not in POSIX (bash-only). #3. fgrep/grep-F are less portable too and need fiddly run-time detection (see AC_PROG_FGREP from autoconf!). Are you going to write the code he needs to select between them..? The OP is explicitly writing a portable shell script, doesn't mention bash at all, so why should you tell him to replace portable constructs with bashisms? – Nicholas Wilson Mar 12 '13 at 11:20
  • @NicholasWilson - #1 - Hoping you won't hit a common problem is ridiculous when you can trivially write scripts to just not allow problems. #2 - correct, that construct is not supported in every shell and if it's not supported in his shell he can do something else better than invoking an external tool and a pipe. #3 fgrep/grep -F are absolutely fine but if you have something against them then use the awk alternative as all 3 alternatives search for strings instead of REs which is important and removes the need to address point 2 at all. – Ed Morton Mar 12 '13 at 14:19
  • @NicholasWilson - I recommend you get the book "Shell Scripting Recipes: A Problem-Solution Approach" by Chris Johnson (http://www.amazon.com/Shell-Scripting-Recipes-Problem-Solution-Approach/dp/1590594711) to help you understand the issues. – Ed Morton Mar 12 '13 at 14:23
  • Cheers. In return, I recommend you read the "Autoconf guide to portable shell" to understand what it's like for who don't live in the bash world, eg how to pick between fgrep/grep-F given broken implementation on some platforms I support. (You're quite right in general about #1, but telling him his "script is all wrong" is going a bit too far.) – Nicholas Wilson Mar 12 '13 at 15:14
  • You seem to be missing the main point that his script IS all wrong. He should simply be using a one-line fgrep or awk solution to search for a STRING, none of that other stuff he's cobbling together to try to escape RE metacharacters so he can then use an RE search using grep but make grep think it's searching for a string! And he needs to quote his shell variable correctly when he does it. – Ed Morton Mar 12 '13 at 16:51
  • Well, if you really feel like writing a portable answer, you're welcome to, but it won't be shorter than pumping the variable through sed and greping for it. When you write portable shell, you just resign yourself to these things to avoid platform quirks, whether or not purists think it's "wrong". Solaris /usr/bin/awk doesn't have "-v", for example, so you can see that these things always become more fiddly than you expect. – Nicholas Wilson Mar 12 '13 at 18:06
  • Doing it the right way is more concise, more robust, faster, and uses fewer resources. What you consider "portable shell" as posted by the user is buggy and error-prone at best. /usr/bin/awk on Solaris is "old, broken awk" and no-one should ever use that for any reason. On Solaris you should use /usr/xpg4/bin/awk. – Ed Morton Mar 12 '13 at 19:18