how to execute a script within a script in shell command

Question

I'm trying to make this script work (what it's supposed to do is search for a string in the last log file of a remote server):

ssh -i $ssh_key_file ubuntu@$1 -t "cd /path/to/logs; grep \"POST api/orders \" `ls -Art | grep info | tail -n 1` -A 2 | grep \"'store_id' => '$2'\" -B 1; bash --login"

this script

`ls -Art | grep info | tail -n 1`

should return to me the last log file, this is an example of it if i run that script alone on the remote server:

$ ls -Art | grep info | tail -n 1
info-2019-04-10.log

further, if I run the exact same script as above by hardcoding the log file name it works just fine:

 ssh -i $ssh_key_file ubuntu@$1 -t "cd /path/to/logs; grep \"POST api/orders \" info-2019-04-10.log -A 2 | grep \"'store_id' => '$2'\" -B 1; bash --login"

I was able to embed this (find last log file) script in other shell scripts and it worked fine, like this one:

ssh -i $ssh_key_file ubuntu@$1 -t "cd /path/to/logs; grep $2 `ls -Art | grep info | tail -n 1`; bash --login"

What am I doing wrong in the original script?

update

using this script

function totCreateOrder()
{
ssh -i "$ssh_key_file" ubuntu@"$1" -t<<EOF
cd /var/www/toters/storage/logs && grep "POST api/orders " "\$(ls -Art | grep info | tail -n 1)" -A 2 \
| grep "'store_id' => '"'$2'"'" -B 1
bash --login
EOF
}

is returning this weird error message

totCreateOrder $prod1as 1006
Pseudo-terminal will not be allocated because stdin is not a terminal.
Welcome to Ubuntu 16.04.5 LTS (GNU/Linux 4.4.0-1067-aws x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

74 packages can be updated.
0 updates are security updates.

New release '18.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.


*** System restart required ***

I looked up what that error means and did this instead

function totCreateOrder()
{
ssh -i $ssh_key_file ubuntu@$1 -t -t<<EOF
cd /var/www/toters/storage/logs && grep "POST api/orders " "\$(ls -Art | grep info | tail -n 1)" -A 2 \
| grep "'store_id' => '"'$2'"'" -B 1
EOF
}

but then i get this error message:

totCreateOrder $prod1as 1006
Welcome to Ubuntu 16.04.5 LTS (GNU/Linux 4.4.0-1067-aws x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

74 packages can be updated.
0 updates are security updates.

New release '18.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.


*** System restart required ***
Last login: Thu Apr 11 03:38:32 2019 from 185.81.141.32
 1"$(ls -Art | grep info | tail -n 1)" -A 2 | grep "'store_id' => '"'1006'"'" -B

update two

i was going over the script line by line.. this worked in the sense that it achieved the following:

logged into remote server
was able to parse ls -Art | grep info | tail -n 1 properly
run this command on remote server: grep "POST api/orders " "$(ls -Art | grep info | tail -n 1)" -A 2

function test() { ssh -i $ssh_key_file ubuntu@$1 -t -t<

it returned this

test $prod1as
Welcome to Ubuntu 16.04.5 LTS (GNU/Linux 4.4.0-1067-aws x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

74 packages can be updated.
0 updates are security updates.

New release '18.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.


*** System restart required ***
Last login: Thu Apr 11 05:18:10 2019 from 185.81.141.85
ubuntu@ip-x-xx-xx:~$ cd /var/www/toters/storage/logs
-Art | grep info | tail -n 1)" -A 2 storage/logs$ grep "POST api/orders " "$(ls
[2019-04-11 04:10:39] production.INFO: POST api/orders [] []
[2019-04-11 04:10:39] production.INFO: array (   'store_id' => '831',   'items' =>    array (     0 =>      array (       'additional_info' => '',       'addons' =>        array (       ),       'quantity' => 1,       'id' => 129369,
),     1 =>      array (       'additional_info' => '',       'addons' =>        array (       ),       'quantity' => 1,       'id' => 133351,     ),     2 =>      array (       'additional_info' => '',       'addons' =>        ..

however things broke when i added the pipe like so:

function test()
{
ssh -i $ssh_key_file ubuntu@$1 -t -t<<EOF
cd /var/www/toters/storage/logs 
grep "POST api/orders " "\$(ls -Art | grep info | tail -n 1)" -A 2 | grep "'store_id' => '"'$2'"'" -B 1
EOF
}

it returns this

 $ test $prod1as 1006
Welcome to Ubuntu 16.04.5 LTS (GNU/Linux 4.4.0-1067-aws x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

74 packages can be updated.
0 updates are security updates.

New release '18.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.


*** System restart required ***
Last login: Thu Apr 11 05:33:22 2019 from 185.81.141.85
ubuntu@ip-10-0-1-39:~$ cd /var/www/toters/storage/logs
-Art | grep info | tail -n 1)" -A 2 | grep "'store_id' => '"'1006'"'" -B 1"$(ls

Kamil Maciorowski · Accepted Answer · 2019-04-11T06:25:04.047

What am I doing wrong in the original script?

The main problem is this: backticks in double-quotes are expanded locally. Normally I would recommend here document, but in your case you need a pseudo terminal and cannot really redirect standard input (which is what here document does). Anyway use $() instead of backticks and escape the $ (or you can stick to backticks and escape them, but $() is preferred in general) or single-quote this fragment. This should pass the entire $( … ) syntax to the remote side. You should also make sure the remote shell gets this in double-quotes, because it's rather obvious you don't want the result to undergo word splitting and pattern matching in this case.

In general one shouldn't parse ls. I guess it would be hard to replace your code with something more robust, unless find on the remote side supports -printf. But I don't know if it does, so I'm not going to improve this part. (Except maybe with this: why reversing the output of ls and taking the last entry? why not the first entry from non-reversed output? does your grep support -m 1?)

grep shouldn't be invoked if cd fails.

You should double-quote variables, unless you know you need not to. :) Not quoting $ssh_key_file or $1 may open a can of… surprises. $2 in your code is somewhat different. From the context I see it should be expanded locally and the result passed to the remote shell. Whatever gets to the remote shell will be parsed. I guess you don't want the result of the local expansion of $2 to undergo variable expansion on the remote side (or do you?). In your original script the remote shell sees the locally expanded $2 in single-quotes inside double-quotes, the outer ones matter. So you should make the locally expanded $2 appear to the remote shell in single-quotes. But then you have literal single quotes embedding this string as parts of the grep pattern, they should be double-quoted. This leads to quoting frenzy, it's tricky.

Still what you're passing to the remote shell is a string that will be parsed, some values of (local) $2 will break the code. Code injection is possible. This is a common issue where commands depending on variables are passed as strings to be parsed, it's rather hard to sanitize this. I hope you have full control over the local $2. If not, it's a vulnerability: anyone who controls it is able to run arbitrary command(s) as ubuntu user on the remote host. E.g. if you run your original code when $2 expands to "& rm -rf ~/ & " (double quotes belong to the variable here), you will be unpleasantly surprised (so don't do it).

The following snippet is totally untested. Your code seems very specific and therefore it's hard to test it.

ssh -i "$ssh_key_file" ubuntu@"$1" -t "
cd /path/to/logs && grep \"POST api/orders \" \"\$(ls -Art | grep info | tail -n 1)\" -A 2 \
| grep \"'store_id' => '\"'$2'\"'\" -B 1
bash --login
"

Notes:

Because of locally expanded $2 and single-quotes that need to get to the remote side, I decided to double-quote the whole command. Putting some part(s) of it in single-quotes might reduce quoting (and escaping) frenzy, but it wouldn't necessarily make the command less cryptic because it would be hard to spot which part is quoted in which way.
$2 still makes code injection possible.

Post-acceptance note

The OP reported the following code worked:

function test()
{
ssh -i "$ssh_key_file" ubuntu@"$1" -t "cd /path/to/logs; grep \"POST api/orders \" \"\$(ls -Art | grep info | tail -n 1)\" -A 2 \
    | grep \"'store_id' => '\"'$2'\"'\" -B 1 \
    ; bash --login"
}

This code fixes the main issue in the same way my code above does. Variations may be important in terms of the syntax(?) but they are secondary to the main issue, I think.

how to execute a script within a script in shell command

update

update two

1 Answers1

Post-acceptance note