0

I have a Bash script in which I call rsync in order to perform a backup to a remote server. To specify that my Downloads folder be backed up, I'm passing "'${HOME}/Downloads'" as an argument to rsync which produces the output:

rsync -avu '/Volumes/Norman Data/Downloads' me@example.com:backup/

Running the command with the variable expanded as above (through the terminal or in the script) works fine, but because of the space in the expanded variable and the fact that the quotes (single ticks) are ignored when included in the variable being passed as part of an argument (see here), the only way I can get it not to choke on the space is to do:

stmt="rsync -avu '${HOME}/Downloads' me@examle.com:backup/"
eval ${stmt}

It seems like there would be some vulnerabilities presented by running eval on anything not 100% private to that script. Am I correct in thinking I should be doing it a different way? If so, any hints for a bash-script-beginner would be greatly appreciated.


** EDIT ** - I actually have a bit more involved use case than. the example above. For the paths passed, I have an array of them, each containing spaces, that I'm then combining into 1 string kind of like

include_paths=(
  "'${HOME}/dir_a'"
  "'${HOME}/dir_b' --exclude=video"
)

for item in "${include_paths[@]}"
  do
    inc_args="${inc_args}" ${item}
  done

inc_args evaluates to '/Volumes/Norman Data/me/dir_a' '/Volumes/Norman Data/me/dir_b' --exclude=video

which I then try to pass as an argument to rsync but the single ticks are read as literals and it breaks after the 1st /Volumes/Norman because of the space.

rsync -avu "${inc_args}" me@example.com:backup/

Using eval seems to read the single ticks as quotes and executes:

rsync -avu '/Volumes/Norman Data/me/dir_a' '/Volumes/Norman Data/me/dir_b' --exclude=video me@example.com:backup/

like I need it to. I can't seem to get any other way to work.


** EDIT 2 - SOLUTION **
So the 1st thing I needed to do was modify the include_paths array to:

  • remove single ticks from within double quoted items
  • move any path-specific flags (ex. --exclude) to their own items directly after the path it should apply to

I then built up an array containing the rsync command and its options, added the expanded include_paths and exclude_paths arrays and the connection string to the remote host.

And finally expanded that array, which ran my entire, properly quoted rsync command. In the end the modified array include_paths is:

include_paths=(
  "${HOME}/dir_a"
  "${HOME}/dir_b"
  "--exclude=video"
  "${HOME}/dir_c"
)

and I put everything together with: 

cmd=(rsync -auvzP)
for item in "${exclude_paths[@]}"
  do  
    cmd+=("--exclude=${item}")
  done

for item in "${include_paths[@]}"
  do
   cmd+=("${item}")
  done

cmd+=("me@example.com:backup/")

set -x
"${cmd[@]}"
Daveh0
  • 952
  • 9
  • 33
  • you can execute like `$("$stmt")` – Digvijay S Mar 16 '20 at 03:41
  • @DigvijayS That won't work at all. The double-quotes tell the shell to treat the entire string -- arguments and all -- as the command name, rather than as a command name followed by arguments. Then the `$( )` makes the shell try to execute anything that manages to print as a second command. – Gordon Davisson Mar 16 '20 at 04:43
  • 1
    Is there a reason to put the command in a variable, rather than just executing it directly? If so, depending on what that reason is, either a function or an array would be a better idea. See [BashFAQ #50: I'm trying to put a command in a variable, but the complex cases always fail!](http://mywiki.wooledge.org/BashFAQ/050) – Gordon Davisson Mar 16 '20 at 04:46
  • @GordonDavisson has the right idea. `eval` is only really necessary in very specific cases such as `eval "$(ssh-agent)"`. – l0b0 Mar 16 '20 at 05:48
  • @GordonDavisson - I guess I don't really need the variable, but see my edit... I *do* have a bit more of an involved use case. Any thoughts on how to pass multiple paths requiring quotes wit spaces? – Daveh0 Mar 16 '20 at 05:51
  • 2
    @Daveh0 Keep them in array form, rather than converting to a single string. Do *not* embed quotes in the array elements (quotes go *around* data, not *in* data), and don't put multiple items in a single array item (as you seem to be doing with "dir_b" and "video"). You can add a prefix to each array element with the `"${array[@]/#/prefix}"` idiom. See [here](https://stackoverflow.com/questions/50710476/bash-need-help-passing-a-a-variable-to-rsync/50710691#50710691) for an example in a very similar situation. – Gordon Davisson Mar 16 '20 at 06:25
  • @GordonDavisson - my problem is that the value of `${HOME}` has a space in it... so if I don't include the single ticks inside the double quotes in the array, the path string ends at the space generates a `No such file or directory` error. – Daveh0 Mar 16 '20 at 06:41
  • Finally got it working using arrays and not embedding quotes just as suggested by @GordonDavisson - thanks for the guidance. OP edited to show final code. – Daveh0 Mar 16 '20 at 14:34

1 Answers1

2

Use an array for the commands/option instead of a plain variable.

stmt=(rsync -avu "${HOME}/Dowloads" me@example.com:backup/)

Execute it using the builtin command

command "${stmt[@]}"

...Or I personally just put the options/arguments in an array.

options=(-avu "${HOME}/Download" me@example.com:backup/)

The execute it using rsync

rsync "${options[@]}"

If you have newer version of bash which that supports the additional P.E. parameter expansion, then you could probably quote the array.

options=(-avu "${HOME}/Download" me@example.com:backup/)

Check the output by applying the P.E.

echo "${options[@]@Q}"

Should print

'-avu' '/Volumes/Norman Data/Downloads' 'me@examle.com:backup/'

Then you can just

rsync "${options[@]@Q}"
Jetchisel
  • 7,493
  • 2
  • 19
  • 18
  • see my edit - I have multiple files/paths I'm passing to `rsync` - this seems to causing problems in that each path needs to be quoted. But i can't even get `opts=(-auv "'${HOME}/Desktop'" me@example.com:backup/) rsync "${opts[@]}"` to work. Gives error: `rsync: link_stat "/Volumes/Norman Data/me/Scripts/backup/'/Volumes/Norman Data/me/Desktop'" failed: No such file or directory (2)`. Any ideas? – Daveh0 Mar 16 '20 at 06:03
  • quoute `${opts[@]}` also `"'${HOME}/dir_b' --exclude=video"` is one argument not two. – Jetchisel Mar 16 '20 at 06:10
  • `echo "${opts[@]}"` does however print `-auv '/Volumes/Norman Data/me/Desktop' me@example.com:backup/` as expected. Stumped. – Daveh0 Mar 16 '20 at 06:10
  • I did - forgot to add when in comment - fixed – Daveh0 Mar 16 '20 at 06:13
  • No issues detected – Daveh0 Mar 16 '20 at 06:21
  • What you posted on your edit has some issues with shellcheck, also why would you want to loop to the array of paths and options? what is the idea behind it when you can just use the array itself without looping? – Jetchisel Mar 16 '20 at 06:24
  • I was missing a quote in my edits. all fixed now. I see what you're saying about looping through the arrays... still even just using a single path (see 1st comment), it still produces a weird path it's trying to find. – Daveh0 Mar 16 '20 at 13:05
  • **re: 1st comment** - removing single ticks from within path arg fixes that small example. This works: `opts=(-auv "${HOME}/Desktop" me@example.com:backup/) rsync "${opts[@]}"` **Final solution** based on suggestions from Jetchisel and Gordon Davisson added to OP. – Daveh0 Mar 16 '20 at 15:54