1

I've written a 'generic' bash script (= generic_script.sh) which serves as a wrapper to start an actual bash script (= actual_script.sh), depending on the parameters given. The stdout of both the generic_script as well as the actual_script should be written to a specific file/folder. The same goes for the stderr of the generic_script as well as the actual_script.

The write path of those two files (stdout and stderr) is dependent on the arguments, which get parsed in the script. Here's a simplified call:

# wrapper-script     job-id job-script                args
./generic_wrapper.sh JOB001 /path/to/actual_script.sh arg1 arg2

generic_wrapper.sh:

#!/bin/bash
{
    # Guarding clauses
    ...

    # Parsing arguments
    stdout_file="/some/path/$1/timestamp.stdout"         # Creating job-id specific folders
    stderr_file="/some/path/$1/timestamp.stderr"

    # Sourcing actual_script.sh
    source "${path}/$2" 1>"${stdout_file} 2>"${stderr_file}"

    # Statistics
    ...

} > "${stdout_file}" 2>"{$stderr_file}" # throws error
echo "${stdout_file}" # returns correct file path

actual_script.sh

#!/bin/bash
echo "Just an example."

However, executing this code returns /wrappers/generic_at_wrapper.sh: line 108: : No such file or directory error. This strongly implies, that at the time of redirection, the variable stdout_file has not been filled. I reckon that this is tied to the order, in which the variables are resolved or the bash is interpreted.

If I echo the value of said variable stdout_file(as well as stderr_file) on the next line, I get the correct value whatsoever, meaning that this is tied to the {} > construct. The redirection method is from this SO-Question.

How can I redirect the stdout and stderr to a file path stored in a variable? The file path variable itself gets calculated in the {} construct and doesn't seem available right after closing the brackets.

Lino
  • 184
  • 1
  • 1
  • 8
  • Are you missing a closing quote here `} > "${stdout_file}`? – Cole Tierney May 31 '21 at 22:23
  • @Cole Tierney: you're absolutely right. But this was solely an issue in my pseudo code above. Fixed it. – Lino May 31 '21 at 22:40
  • @Lino: You are **using** `stdout_file` outside the `{...}` sequence, but define it inside. When bash sets up the outside redirection, the variables are still undefined. Your redirection is expanded to `> ""`, and this is what the error message says. – user1934428 Jun 01 '21 at 08:03

2 Answers2

2

The {} > ... construct tries running the code in {} after > .... Bash needs to know the output redirection before running the command(s). Have you tried setting stdout_file and stderr_file before the code that you want the output of redirected?

That is, your code should be:

stdout_file="/some/path/$1/timestamp.stdout"         # Creating job-id specific folders
stderr_file="/some/path/$1/timestamp.stderr"
{
   # ...content subject to redirections here...
} >"$stdout_file" 2>"$stderr_file"

You may also find >&1, 2>&2 useful, rather than passing in folders.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
j1mbl3s
  • 994
  • 3
  • 16
  • @CharlesDuffy I've updated my answer to be authoritative. I can't comment on questions yet, and without definitively knowing that what I am saying is correct I default to guessing. Thank you for confirming :) – j1mbl3s May 31 '21 at 22:42
  • BTW, I'm not quite sure what you mean re: `>&1` or `2>&2` -- both those operations make a file descriptor a copy of its prior value, and thus have no effect at all. – Charles Duffy May 31 '21 at 22:51
  • I meant `>&1 2>&2` to be used for the redirection of the `actual_script.sh` output. So the full command could be run with, for example: `./generic_wrapper.sh > ./stdout.log 2> ./stderr.log`. But I see that the OP wanted to add some additional information to the filename, like timestamp, so it may be redundant. – j1mbl3s May 31 '21 at 23:05
  • 1
    Sure. That said, I assume that when the OP says "timestamp.stdout", their real use case is something more like `$(date +%Y%m%dT%H%M%S).stdout`. They'd need a separate shell script as a wrapper to automate that, so it's reasonable to want to do it internally. – Charles Duffy May 31 '21 at 23:10
2

If you want to redirect for the entirety of your code, instead of using blocks, use the exec command. That is:

stdout_file="/some/path/$1/timestamp.stdout"         # Creating job-id specific folders
stderr_file="/some/path/$1/timestamp.stderr"
exec >"$stdout_file" 2>"$stderr_file"

# ...all code below this point has stdout going to stdout_file, and stderr to stderr_file
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • I'd prefer this method of redirection, as it immediately shows, that a redirection takes place without having to read the entire script/block. I've removed the block `{}`, created the stdout and stderr variables before doing the `exec` and the timestamp specific files get created. Surprisingly, there is no output (to either stdout or stderr) before I have sourced the `actual_script.sh`. So doing an `echo "test"` immediately after the exec will not be visible in the output. According to the example given, the first line written is `Just an example.` Any thoughts on this? – Lino Jun 04 '21 at 14:11
  • I'd need to see a reproducer. With some commands you might see libc's buffering behaviors causing writes to be deferred (until there's enough content to be able to do a larger, lower-overhead write) when stdout is going somewhere other than a tty, but `echo` is not such a command. – Charles Duffy Jun 04 '21 at 14:23
  • The cause was actually much simpler. I forgot to adjust the redirection on the source, which is in fact not needed at all using `exec` to redirect. See explanation in this question: https://stackoverflow.com/questions/67839242/incomplete-redirection-of-stdout-and-stderr-using-exec – Lino Jun 04 '21 at 15:13