1

This is a follow-up to this question:

How do I get both STDOUT and STDERR to go to the terminal and a log file?

In short: I want to run a command and store both, STDOUT and STDERR in one log file (to keep time correlation) but I need STDERR output still to be printed on STDERR.

Motivation: there are tools which run a command and treat STDOUT and STDERR differently - e.g. printing STDERR in a different color or print STDERR when the command returns non-zero.

So I'd like to have a way to store all output in one log file but preserve the distinction between STDOUT and STDERR (as well as the return code).

log-output --file=command.log -c "make stuff-with-stderr"

From what I found in the above links answers there are at least two different approaches:

the_cmd 1> >(tee stdout.txt ) 2> >(tee stderr.txt >&2 )

will store STDOUT and STDERR in separate files, thus loosing time correlation. And unfortunately both STDOUT and STDERR will be printed on STDOUT only.

script -e -B build.log -c "the_cmd"

will store both STDOUT and STDERR in one file, keeping time correlation but still prints both STDOUT and STDERR on on STDOUT only.

So none of those approaches meets my requirements. Is there something else?

frans
  • 8,868
  • 11
  • 58
  • 132

1 Answers1

1

Edit:

Using my approach and your command from the comment below (pip3 install -U pytest) in a bash shell you get your desired outcome:

(((pip3 install -U pytest | tee -a log.txt) 3>&1 1>&2 2>&3 | tee -a log.txt) 3>&1 1>&2 2>&3)

The entries in log.txt are always in the correct order (same as executing pip3 install -U pytest in a terminal) and the output is printed to the terminal correctly (separate streams for stdout and stderr). I suspect a better solution exists, but this appears to be a 'workable' solution to the problem.

Original answer:

There's probably a less awkward solution than this, but you could tee stdout to log.txt, then swap stdout with stderr and tee the 'new' stdout (stderr) to log.txt, then swap stderr back with stdout, then print both streams to terminal (similar to the example here):

((({ echo "test_out"; echo "test_err" 1>&2; } | tee -a log.txt) 3>&1 1>&2 2>&3 | tee -a log.txt) 3>&1 1>&2 2>&3)
test_out
test_err

cat log.txt
test_out
test_err

# show stdout/stderr are still separate streams:
((({ echo "test_out"; echo "test_err" 1>&2; } | tee -a log.txt) 3>&1 1>&2 2>&3 | tee -a log.txt) 3>&1 1>&2 2>&3) 1>std.out 2>std.err

cat std.out
test_out

cat std.err
test_err
jared_mamrot
  • 22,354
  • 4
  • 21
  • 46
  • looks promising, but doesn't fully work yet. E.g. I get stdout printed twice. And trying to pipe `stdout` to `/dev/null` I still get printed everything except the doubled lines from `stdout`. So it seems like the commands `stdout` is printed to both, `stdout` and `stderr`, thus the duplicate lines. what shell do you use? – frans Dec 06 '22 at 12:19
  • Hm, and trying your `echo` example using `bash` I don't see duplicate lines, but it looks like I have to leave away the curly braces when using non-built-in commands. I try `pip3 install -U pytest`, which gives me some `stdout` and `stderr` (due to an outdated version of `pip` installed) – frans Dec 06 '22 at 12:25
  • Oh! Sorry, I wasn't clear, this is specific to bash (I saw the bash tag and assumed that's the shell you were using). For your pip3 example, you are right that you don't need the curly braces, i.e. `(((pip3 install -U pytest | tee -a log.txt) 3>&1 1>&2 2>&3 | tee -a log.txt) 3>&1 1>&2 2>&3)` returns your desired outcome. – jared_mamrot Dec 06 '22 at 22:09