How to remove carriage return from a variable in shell script

Question

I am new to shell script. I am sourcing a file, which is created in Windows and has carriage returns, using the source command. After I source when I append some characters to it, it always comes to the start of the line.

test.dat (which has carriage return at end):

testVar=value123

testScript.sh (sources above file):

source test.dat
echo $testVar got it

The output I get is

got it23

How can I remove the '\r' from the variable?

When you've written a Linux script in Windows, you need to convert the whole file so that all of the carriage returns are removed before running it, otherwise you will run into all sorts of problems. Attempting to replace individual carriage returns might fix some problems, but others will appear. For me it was often with messages about lines that were truncated. Thus, it is best to convert the whole file using a program like `dos2unix`. — Dave F, Apr 07 '21 at 21:49
Its an old one ofcourse, but this oneliner should do too. `echo ${testVar%'\r'}` — AP22, Jun 30 '21 at 02:17

score 92 · Answer 1 · answered Mar 20 '13 at 10:42

92

yet another solution uses tr:

echo $testVar | tr -d '\r'
cat myscript | tr -d '\r'

the option -d stands for delete.

answered Mar 20 '13 at 10:42

Nik O'Lai

3,586
1
15
17

This helps but it's removing the last word/character for me. – Clover Oct 16 '22 at 09:21
@Clover, can u please provide an example? I think there is something else hiding – Nik O'Lai Oct 16 '22 at 12:32

score 32 · Answer 2 · edited Aug 03 '18 at 21:24

32

You can use sed as follows:

MY_NEW_VAR=$(echo $testVar | sed -e 's/\r//g')
echo ${MY_NEW_VAR} got it

By the way, try to do a dos2unix on your data file.

edited Aug 03 '18 at 21:24

Benjamin W.

46,058
19
106
116

answered Mar 20 '13 at 10:06

Xavier S.

1,147
13
33

I implemented this and it deletes every "r" not just carriage returns. – ffuentes Mar 18 '21 at 15:43
Are you sure you kept the \ ? – Xavier S. Mar 20 '21 at 20:27

Benjamin W. · Answer 3 · 2019-12-24T01:23:02.993

25

Because the file you source ends lines with carriage returns, the contents of $testVar are likely to look like this:

$ printf '%q\n' "$testVar"
$'value123\r'

(The first line's $ is the shell prompt; the second line's $ is from the %q formatting string, indicating $'' quoting.)

To get rid of the carriage return, you can use shell parameter expansion and ANSI-C quoting (requires Bash):

testVar=${testVar//$'\r'}

Which should result in

$ printf '%q\n' "$testVar"
value123

edited Dec 24 '19 at 01:23

answered Aug 03 '18 at 14:39

Benjamin W.

46,058
19
106
116

I needed \n instead of \r ... for some reason – David Goodwin Nov 05 '18 at 13:33
@DavidGoodwin Weird. What does the output of `printf '%q\n' "$testVar"` look like? – Benjamin W. Nov 05 '18 at 19:54
$'include:(\n .......blah blah .... )\n)' (My multiline string was made from ```sh VAR=$(cat < – David Goodwin Nov 07 '18 at 09:10
@DavidGoodwin But do you even have carriage returns you want to remove? – Benjamin W. Nov 07 '18 at 14:59
Aaaaaaaaaaah :lightbulb: etc. Thanks :) – David Goodwin Nov 08 '18 at 16:30
Thank you for this thread. I spent one hour pulling my hair because I was reading a Windows file into a Bash variable and the result was total nonsense. `testVar=${testVar//$'\r'}` did the trick. – user3262284 Jan 11 '23 at 22:05

score 8 · Answer 4 · edited Mar 20 '13 at 10:49

8

use this command on your script file after copying it to Linux/Unix

perl -pi -e 's/\r//' scriptfilename

edited Mar 20 '13 at 10:49

glenn jackman

238,783
38
220
352

answered Mar 20 '13 at 10:01

Vorsprung

32,923
5
39
63

Thanks a ton for `perl`. I couldn't download `dos2unix`, `tr` wouldn't work and what not! Thanks a lot! – Mrigank Pawagi Jan 26 '18 at 16:55
1

I used `perl -p -e 's/\r\n$//' ` to trim `\r\n` from only the end of the input. – Gordon Bean Jun 13 '19 at 20:20

score 6 · Answer 5 · edited Aug 03 '18 at 21:24

6

Pipe to sed -e 's/[\r\n]//g' to remove both Carriage Returns (\r) and Line Feeds (\n) from each text line.

edited Aug 03 '18 at 21:24

Benjamin W.

46,058
19
106
116

answered Aug 29 '16 at 13:21

mcandre

22,868
20
88
147

4

You cannot remove `\n` using sed like that. See http://sed.sourceforge.net/sedfaq5.html#s5.10 – Bryan May 15 '17 at 10:14

score 5 · Answer 6 · edited Feb 27 '18 at 23:18

5

for a pure shell solution without calling external program:

NL=$'\n'    # define a variable to reference 'newline'

testVar=${testVar%$NL}    # removes trailing 'NL' from string

edited Feb 27 '18 at 23:18

t j

7,026
12
46
66

answered Feb 27 '18 at 22:59

The O.G. X

51
1
1

The problem is the carriage return, not the newline, though. – Benjamin W. Aug 03 '18 at 14:43
so NL=$'\n' and NR=$'\r' and replace both, nothing happens on a no find condition – user176495 Nov 28 '18 at 01:26

How to remove carriage return from a variable in shell script

6 Answers6

Linked

Related