Bourne/POSIX shell parameter splitting

Question

Assume I have some utility that can take a number of options, each followed by a filename. E.g., I could call it as myutil, myutil -o somefile, myutil -p anotherfile, myutil -o somefile -p anotherfile, etc.... I want to write a wrapper POSIX shell script that is able to call myutil with arbitrary combinations of options (depending on some conditions internal to the wrapper script, which aren't relevant to this question).

I thought of doing something like:

#!/bin/sh
file1=somefile
file2=anotherfile

if [ somecriteria = true ]; then
  OPT1="-o $file1"
fi

if [ othercriteria = true ]; then
  OPT2="-p $file2"
fi

myutil $OPT1 $OPT2

This works great—as long as neither filename has spaces: Assuming both ifs are true, myutil gets $1 = [-o], $2 = [somefile], $3 = [-p], and $4 = [anotherfile]. However, if there are spaces, e.g., if file1="some file", $1 = [-o], $2 = [some], $3 = [file], etc. Of course, what I want is for $2 = [some file].

Putting another set of quotes around the filename in OPT1 and OPT2 doesn't help; e.g., if I change it to OPT1="-o \"$file1\"", that just gets me $2 = ["some] and $3=[file"]. And putting in quotes around $OPT1 and $OPT2 in the call to myutil doesn't work either: if I do that, $1 = [-o some file].

So, is there some trick to get this working, or some other approach that would do what I want? I'd like this to stick to standard shell features, so no bash-isms or ksh-isms, please :) See this for a description of what's in the standard.

if you set your variable like this: `var=true`, then you can write your condition like this: `if $var; then do something; fi` — glenn jackman, Jan 20 '11 at 16:00
The `if` statements are just meant to be more like pseudocode :) `[ somecriteria = true ]` compares the literal strings `somecriteria` and `true`, and will always be false—the actual code definitely isn't like that :) — Dave Huang, Jan 20 '11 at 17:30

Dave Huang · Answer 1 · 2011-01-21T09:37:10.067

After messing with it more, I found another approach that seems to do what I want:

#!/bin/sh
file1="some file"
file2=anotherfile

if [ somecriteria = true ]; then
  OPT1="$file1"
fi

if [ othercriteria = true ]; then
  OPT2="$file2"
fi

myutil ${OPT1:+-o "$OPT1"} ${OPT2:+-p "$OPT2"}

The construct ${parameter:+word} will be substituted with word if ${parameter} is set; if it's not set, it goes away. So if $OPT1 is unset, ${OPT1:+-o "$OPT1"} disappears—notably, it doesn't turn into an empty string in argv. If $OPT1 is set to some file, the above expression is substituted with -o "some file", and myutil gets $1 = [-o], $2 = [some file] as I want.

Note that myutil ${OPT1:+-o} "$OPT1" ${OPT2:+-p} "$OPT2" does not do exactly what I want, because if $OPT1 is unset, the -o goes away, but "$OPT1" turns into an empty string—$1 = [], $2 = [-p], $3 = [anotherfile]

(Edited per Dennis's suggestion)

+1 Nice solution! Try doing it with one brace expansion per option: `${OPT1:+-o "$OPT1"}`. It seems to work for me. — Dennis Williamson, Jan 20 '11 at 18:19

SiegeX · Accepted Answer · 2011-01-21T01:27:38.557

2

Seems like you found a decent POSIX solution. You could, however, use set to maintain a call to your program as myutil "$@". Your solution gets a bit unwieldy as the number of possible parameters grow.

#!/bin/sh
file1=somefile
file2=anotherfile

if [ somecriteria = true ]; then
  set -- "-o" "$file1"
fi

if [ othercriteria = true ]; then
  set -- "$@" "-p" "$file2"
fi

myutil "$@"

Example

#!/bin/sh

file1="some file"
file2="another file"

# Default to '1' if not overwritten
: ${x:=1}
: ${y:=1}

if [ $x -eq 1 ]; then
  set -- "-o" "$file1"
fi

if [ $y -eq 1 ]; then
  set -- "$@" "-p" "$file2"
fi

printf "[%s]" "$@"
echo

Output

$ x=0 y=0 ./opt.sh
[]
$ x=0 y=1 ./opt.sh
[-p][another file]
$ x=1 y=0 ./opt.sh
[-o][some file]
$ x=1 y=1 ./opt.sh
[-o][some file][-p][another file]

edited Jan 21 '11 at 01:27

answered Jan 20 '11 at 09:02

SiegeX

135,741
24
144
154

Thanks, but I specifically requested no bash-isms. Arrays are a non-standard bash extension. See http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html for info on what's included in the standard. – Dave Huang Jan 20 '11 at 10:25
@Dave updated my answer to give you POSIX solution that more closely matches your original code. – SiegeX Jan 21 '11 at 01:23
Yeah, this solution does seem easier to manage than the one I found. Thanks! :) – Dave Huang Jan 21 '11 at 09:41

lecodesportif · Answer 3 · 2011-01-21T22:33:50.900

2

First of all, you'll have to quote the options in this line sh myutil.sh "$OPT1" "$OPT2"

And here's a working implementation with no particular -isms that uses getopts on the myutil.sh side.

This script calls myutil.sh:

#!/bin/sh
somecriteria=true
othercriteria=true
file1="some file"
file2="other file"

if [ $somecriteria = true ]; then
  OPT1="-o$file1"
fi

if [ $othercriteria = true ]; then
  OPT2="-p$file2"
fi

sh myutil.sh "$OPT1" "$OPT2"

And this is what myutil.sh could look like:

#!/bin/sh
OPTIND=1
while getopts "o:p:" opt; do
  case "$opt" in
    o)  file1=$OPTARG
        ;;
    p)  file2=$OPTARG
        ;;
  esac
done
shift $((OPTIND-1))

echo 'File 1: "'$file1'"'
echo 'File 2: "'$file2'"'

As you can see in the output of myutil.sh, spaces in the filenames are preserved:

File 1: "some file"
File 2: "other file"

edited Jan 21 '11 at 22:33

answered Jan 20 '11 at 11:04

lecodesportif

10,737
9
38
58

Ah, I guess I sort of made it sound like `myutil` was something I wrote and have control over—it's not actually *my* utility; it's a third-party util. While it is open source, and I could patch its argument parsing to work with your suggested solution, it's not practical for me to tell everyone that my script doesn't work with the stock utility and they have to patch it. – Dave Huang Jan 20 '11 at 17:37
Not quite. The correct quoting has to be done on the side of *your* script that calls myutil. But never mind. – lecodesportif Jan 21 '11 at 22:31

mtraceur · Answer 4 · 2015-04-16T20:55:10.270

I think your own solution using ${OPT1+-o "$OPT1"} is a good one, and off the top of my head I don't see any problems with it in this usecase, but there's another approach using eval that no one mentioned, which is even closer to your original code:

#!/bin/sh
FILE1='some file'
FILE2='another file'

if [ somecriteria = true ]; then
 OPT1="-o '$FILE1'"
fi

if [ othercriteria = true ]; then
  OPT2="-p '$FILE2'"
fi

eval myutil "$OPT1" "$OPT2"

This will produce what you want.

But you need to be careful in case your filenames contain single-quotes as part of the literal filename string.

If you know exactly what your filenames look like when writing the script, just ensure your filename's quote literals don't break out of whatever escaping you put around the file name.

But this is even more important when you're handling user input, or otherwise getting input from the environment. - for example, if $FILE1 is defined as abc'; /tmp/malicious_program ' and then you do the eval, it will parse the myutil line into:

myutil -o 'abc'; /tmp/malicious_program '' -p 'another file'

..which is two separate commands, and could be a massive security hole, depending on how exactly this script is being executed relative to the entity which created /tmp/malicious_program and set $FILE1.

In those cases, if you are willing to introduce a dependency on sed, you can do something like this first:

FILE1=\'`printf %s "$FILE1" | sed "s/'/'\\\\''/g"`\'

..this will produce a nice, single-quote escaped filename, with any sungle-quotes inside it properly escaped too.

Since in Bourne/POSIX shell nothing but a single quote can "break out of" a single quoted string, this is why I use single-quoting in my example. Double quote escaping things this way is possible, but your sed command has to be much more complex since there's several other characters you need to escape inside double quotes (off the top of my head: instead of just escaping singe quotes, you escape double quotes, backslashes, backticks, dollar signs, and possibly other stuff I'm not thinking of).

P.S.: because I found this "wrap in shell escaping, then eval later" approach useful in several cases, in at least one where it was downright necessary, I wrote a tiny C program (and equivalent shell function using sed) which wraps all of its arguments in single quote shell escaping as described above, in case anyone wants to use it instead of having to implement your own: esceval

score 1 · Answer 5 · answered Jan 20 '11 at 15:51

1

Why note simple quote instead of double quote ?

if [ somecriteria = true ]; then
  OPT1="-o '$file1'"
fi

if [ othercriteria = true ]; then
  OPT2="-p '$file2'"
fi

answered Jan 20 '11 at 15:51

Flyounet

105
1
6

Because that doesn't work either. With single quotes, $1 = [-o], $2 = ['some], $3 = [file'] – Dave Huang Jan 20 '11 at 17:27

score 0 · Answer 6 · answered Jan 20 '11 at 09:06

0

Possible implementation is below, bash arrays tutorial is here.

#!/bin/sh

function myutil {
  local a1=$1; shift;
  local a2=$1; shift;
  local a3=$1; shift;
  local a4=$1; shift;

  echo "a1=$a1,a2=$a2,a3=$a3,a4=$a4"
}

file1="some file"
file2="another file"

OPT1=(-o "$file1")
OPT2=(-p "$file2")

myutil "${OPT1[@]}" "${OPT2[@]}"

answered Jan 20 '11 at 09:06

bobah

18,364
2
37
70

1

You shouldn't use the keyword `function`, especially if you're going to use `#!/bin/sh` – SiegeX Jan 20 '11 at 09:09
@SiegeX - what makes you state this? this syntax is more consistent with how arguments are treated in the shell, and is closer to that of Perl which many people use as a complimentary tool for bulky text data processing. – bobah Jan 20 '11 at 09:35
2

Sorry, I should have been more clear. The reason is because `function` is not POSIX compliant. See [this question](http://stackoverflow.com/questions/4654700/what-are-the-parentheses-used-for-in-a-bash-shell-script-function/4654730#4654730). Instead, use `myutil ()` – SiegeX Jan 20 '11 at 09:48
Arrays in general aren't POSIX-compliant either... I'm looking for a solution that doesn't require bash. – Dave Huang Jan 20 '11 at 10:27
1

`/bin/sh` doesn't generally support arrays. – Dennis Williamson Jan 20 '11 at 18:16

score 0 · Answer 7 · 2011-01-20T10:30:36.973

0

Use getopts http://mywiki.wooledge.org/BashFAQ/035

"The POSIX shell (and others) offer getopts which is safe to use instead."

edited Jan 20 '11 at 10:30

answered Jan 20 '11 at 09:07

the problem is not to parse options, but to have file name containing space passed as a single option – bobah Jan 20 '11 at 09:11

Bourne/POSIX shell parameter splitting

7 Answers7

Example

Output