Retaining trailing newlines in function results in bash

Question

The bash script somehow loses all trailing newlines, but I have no idea where, or how to keep them

#!/bin/bash
function rep {
    FIRST=${1//&nbsp;/" "}
    SECOND=${FIRST//&t;/"   "}
    echo "${SECOND//&nl;/"
"}"
}

if [ "$7" == "" ]
then
    cd "$(rep $6)"
    th sample.lua -checkpoint cv/"$(rep $1})"/"$(rep $2)" -length $3 -temperature $4 -sample 1 > samples/"$(rep $1)"/Sample-$5.txt
else
    cd "$(rep $7)"
    th sample.lua -checkpoint cv/"$(rep $1)"/"$(rep $2)" -length $3 -temperature $4 -sample 1 -start_text "$(rep $5)" > samples/"$(rep $1)"/Sample-$6.txt
fi

In this script the function rep replaces tags I have made in input strings that replace to spaces, tabs, and newlines. I'm pretty sure any whitespace and newline characters get trimmed off.

I've looked at solutions like this, https://stackoverflow.com/a/15184414/5332233, but I haven't been able to get it to work in the code.

EDIT - Here's the function when I tried to add a dummy character to the variable

function rep {
    FIRST="${1//&nbsp;/" "}"
    SECOND="${FIRST//&t;/"  "}"
    THIRD="${SECOND//&nl;/"
"}"
    a=$(printf $THIRD; printf x); echo ${a%x}
}

Here's another try, this still gets rid of all of the trailing newlines

#!/bin/bash
function rep {
    IN="$1x"
    FIRST="${IN//&nbsp;/" "}"
    SECOND="${FIRST//&t;/"  "}"
    THIRD="${SECOND//&nl;/"
"}"
    echo "${THIRD%x}"
}

echo "$(rep $5)"

if [ "$7" == "" ]
then
    cd "$(rep $6)"
    th sample.lua -checkpoint cv/"$(rep $1})"/"$(rep $2)" -length $3 -temperature $4 -sample 1 > samples/"$(rep $1)"/Sample-$5.txt
else
    cd "$(rep $7)"
    th sample.lua -checkpoint cv/"$(rep $1)"/"$(rep $2)" -length $3 -temperature $4 -sample 1 -start_text "$(rep $5)" > samples/"$(rep $1)"/Sample-$6.txt
fi

Yes; command substitutions strip *trailing* new lines. The linked answer provides several workarounds for when that isn't desirable; which did you try? — chepner, Aug 01 '17 at 21:52
I've tried appending a character and removing it when I'm using it in the command, but I wasn't able to get that work and the other two I couldn't figure out the right way to implement it — Butterscotch, Aug 01 '17 at 21:53
I fixed the latest attempt because I messed that one up, but now it only keeps the first trailing newline — Butterscotch, Aug 01 '17 at 22:36
Nevermind, I messed up when testing that one, it still doesn't work — Butterscotch, Aug 01 '17 at 22:42
Okay, I've figured out what the issue is, it's the function, when I tried setting all of the arguments with those lines for each one, it worked. — Butterscotch, Aug 01 '17 at 23:08
Adding the `x` doesn't work if you strip it off **before** the output goes through the command substitution; it has to be *after* for that workaround to have any effect. — Charles Duffy, Aug 01 '17 at 23:35
btw, `==` isn't guaranteed to work in `[ ]`; the POSIX-compliant string comparison operator is `=`. And you've got a ton of unquoted expansions; http://shellcheck.net/ will find and alert on them. — Charles Duffy, Aug 01 '17 at 23:36
And the `function` keyword is a bashism that adds no value over the more-compatible, standardized, POSIX-compliant function declaration syntax (`funcname() { ...body...; }`); it's there for compatibility with ksh, but is best avoided. — Charles Duffy, Aug 01 '17 at 23:37
You have to strip the extra `x` from the *result* of the command substitution. `foo () { echo $'foo\n\n'; }; y=$(foo); y=${y%x}`. — chepner, Aug 01 '17 at 23:47

Charles Duffy · Accepted Answer · 2017-08-01T23:45:44.943

See BashFAQ #6 for various ways to perform indirect assignment. This lets you run your function in the same shell where you need its output to be present in a variable, so there's no need for command substitution (and you avoid paying the performance penalty of fork()ing for a subshell besides).

rep() {
  local out=$1 in=$2 data
  data=${in//&nbsp;/ }
  data=${data//&t;/$'\t'}
  data=${data//&nl;/$'\n'}
  printf -v "$out" %s "$data"
}

input='hello &nbsp; world&nl;'
rep result_var "$input"
printf '%s\n' "<${result_var}>"

...properly emitting as output:

<hello   world
>

Note that assignments are implicitly exempt from string-splitting and glob expansion, so the right-hand side don't need to be quoted unless their contents would be parsed as multiple words before expansions are honored.

Thanks a lot! I appreciate all the help you gave, this is a good answer and it's better code than I could ever do in Bash. — Butterscotch, Aug 01 '17 at 23:53

Retaining trailing newlines in function results in bash

1 Answers1