0

I need to quote end of line and using this solution to replace end of line which working perfectly when file has multiple lines.

Problem arises when I try to use this method with files that have no \n or have single line ending with \n. In this case sed does not replace anything, even second s/ command does not work. Also looks like last end of line is never replaced.

My command is:

sed ':a;N;$!ba;s/\n/\\n/g;s/a/_/g' <file>

(replace \n to \\n and replace a to _ for example).

I use hexdump -C to display files here:

There are tests I made:

$ # Test1. single line without \n
$ echo -n 'abc' | sed ':a;N;$!ba;s/\n/\\n/g;s/a/_/g' | hexdump -C
00000000  61 62 63                                          |abc|
# but expected "_bc"

$ # Test2. single line with \n
$ echo 'abc' | sed ':a;N;$!ba;s/\n/\\n/g;s/a/_/g' | hexdump -C
00000000  61 62 63 0a                                       |abc.|
# but expected "_bc\n"

$ # Test3. two lines
$ echo -e 'abc\n' | sed ':a;N;$!ba;s/\n/\\n/g;s/a/_/g' | hexdump -C
00000000  5f 62 63 5c 6e 0a                                 |_bc\n.|
# but expected "_bc\n\n"

Questions: Why second s/ does not replace anything in Test1 and Test2? Is there any method to fix replacing of s/\n/\\n/ and s/a/_/g in all this tests?

P.S.: I don't want some workaround like adding newline at end of stream before sed processing and removing it after.

EDIT: looks like N command does not read single line, even if it followed by '\n'. Any idea how to fix it?

EDIT2: Seems like this is expected behavior of N command. Is there any method on how to replace last line ending?

Community
  • 1
  • 1
loentar
  • 5,111
  • 1
  • 24
  • 25
  • I used this recently: sed '{:q;N;s/\n/_/g;t q}' – svante Aug 16 '13 at 10:53
  • Have you tried separating the sed parts? `sed ':a;N;$!ba' | sed 's/\n/\\n/g;s/a/_/g'` or similar. – svante Aug 16 '13 at 11:09
  • This does not help, because it is equal to `cat | sed 's/\n/\\n/g;s/a/_/g'`. Joining lines using `':a;N;$!ba'` is working just within one sed process. – loentar Aug 16 '13 at 11:13
  • `sed ':a;N;$!ba;s/\n/\\n/g' | sed 's/a/_/g'` ? Or I'm out of suggestions, sorry. – svante Aug 16 '13 at 11:16
  • Thanks, but this does not replace end of line. Unfortunately, this does not solve the problem itself. – loentar Aug 16 '13 at 11:24

3 Answers3

3

This might work for you (GNU sed):

sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' file

or:

sed ':a;$!N;$!ba;s/\n/\\n/g;y/a/_/' file

Logically it makes sense: one cannot get the next line if you are on the last line.

potong
  • 55,640
  • 6
  • 51
  • 83
  • Thank you for your answer. I understand the logic. Unfortunately it does not replace last end of line to '\\n' (Test2 and Test3). Possibly there is no solution to make `N` read don't skip last line. – loentar Aug 16 '13 at 19:09
  • As it cannot read a line that does not exist just add one! `sed ':a;$!{N;ba};G;s/\n/\\n/g;s/a/_/g' file – potong Aug 16 '13 at 22:24
  • Unfortunately, this always adds new line into source stream, so Test1 is failed. Also it does not remove last end line, so all tests are failed. – loentar Aug 19 '13 at 06:06
2

Can you use perl instead?

$ echo -n "abc" | perl -pe 's/\n$/\\n/;' -pe 's/a/_a/;' | od -x
0000000 615f 6362
0000004
$ echo "abc" | perl -pe 's/\n$/\\n/;' -pe 's/a/_a/;' | od -x
0000000 615f 6362 6e5c
0000006
$ echo -e 'abc\n' | perl -pe 's/\n$/\\n/;' -pe 's/a/_a/;' | od -x
0000000 615f 6362 6e5c 6e5c
0000010
devnull
  • 118,548
  • 33
  • 236
  • 227
  • 1
    Bash is strongly preferred. Don't want to add new dependency just for one script of >30. – loentar Aug 16 '13 at 11:11
  • @loentar Do you have a character that doesn't appear in your input :) – devnull Aug 16 '13 at 11:17
  • I'm sorry, had typos in "expected results", your second regex should look like `'s/a/_/;'`, also it can be combined into single argument: `perl -pe 's/\n$/\\n/g;s/a/_/g'`. Thank you for answer anyway. – loentar Aug 16 '13 at 11:34
0

Sad, but I don't find any method to prevent sed from adding new line to output when N command is used. I didn't want to some workaround to like add new line before processing and removing it after, but seems like there is no other method.

This is the final solution with workaround:

sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' <<< "$VAR" | tr -d '\n'

<<< like echo adds new line into input string, tr removes it.

With that line all test are OK:

$ echo -e "abc" | sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' | tr -d '\n' | hexdump -C
00000000  5f 62 63                                          |_bc|

$ echo -e "abc\n" | sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' | tr -d '\n' | hexdump -C
00000000  5f 62 63 5c 6e                                    |_bc\n|

$ echo -e "abc\n\n" | sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' | tr -d '\n' | hexdump -C
00000000  5f 62 63 5c 6e 5c 6e                              |_bc\n\n|

P.S.: thanks potong for figuring out the :a;$!{N;ba}.

P.P.S.: If someone will give working answer without tr, I will re-accept it.

loentar
  • 5,111
  • 1
  • 24
  • 25