3

I have the following lftp script to copy files from a remote to local:

env TERM=dumb script -a $LOGSTDOUT -c "$(cat <<- EOF
    lftp $PROTOCOL://$URL -u ${USER},${PASS} << EOFF
    set dns:fatal-timeout never
    set sftp:auto-confirm yes
    set mirror:use-pget-n 50
    set mirror:parallel-transfer-count 2
    set mirror:parallel-directories yes
    set mirror:include-regex $REGEX
    set log:enabled/xfer yes
    set log:file/xfer $LOG
    set xfer:use-temp-file yes
    set xfer:temp-file-name *.lftp
    mirror -c -v --loop --Remove-source-dirs "$REMOTEDIR" "$LOCALDIR"
    quit
    EOFF
EOF
)"

I am capturing terminal output with the script(1) utility. The env TERM=dumb is just a random piece of code I found to disable ANSI escape codes.

My problem is that the line breaks of the output log file get quiet mangled. It seems to be using CR and LF. I discovered more information here and it seems this is by design. Though I'm not sure how to fix it.

These line endings cause issues when viewing the logs in lnav:

enter image description here

The reason for this becomes quickly apparent upon inspecting the raw text:

enter image description here

I have thought of some potential options, but not sure how to implement:

  1. Fix the output of the script(1) utility so that single CR are converted to LF. Maybe this can be achieved with piping or some argument?

  2. A hack for lnav to treat CR as LF when displaying in the GUI.

Does anyone know how I can fix these line breaks so it shows correctly in lnav?

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
mwsmws22
  • 127
  • 2
  • 11
  • most probably the command [dos2unix](https://linux.die.net/man/1/dos2unix) can fix it for you. – pynexj Jun 03 '23 at 14:51
  • @pynexj The problem is it isn't a static file. I don't think I can apply dos2unix with the script(1) utility with a constantly changing log file. But correct me if I'm wrong. – mwsmws22 Jun 03 '23 at 15:26
  • save script's output to a file and then run dos2unix on the file. – pynexj Jun 03 '23 at 15:36
  • 1
    add a filter where appropriate `... | tr -d '\012'` . Good luck. – shellter Jun 03 '23 at 15:48

1 Answers1

2

Try replacing

... script -a $LOGSTDOUT ...

with

... script -a >(tr -d '\r' >"$LOGSTDOUT") ...
pjh
  • 6,388
  • 2
  • 16
  • 17
  • 1
    Thank you! This worked great. I also appreciate the links! I ended up using a modified version of the above: `>(tr '\r' '\n' | cat -s >> "$LOGSTDOUT")`. The `tr '\r' '\n'` is for replacing CR with LF. But this creates extra blanks links, so I pipe it to `cat -s`. Finally, used `>>` instead of `>` to append to the file, not overwrite it. – mwsmws22 Jun 04 '23 at 06:34