Data transformation using sed

Question

I have a file like:

A
B
C
D

E
F
G
H

I
J
K
L

and I want it to come out like

A,B,C,D
E,F,G,H

I'm assuming I'd use sed, but actually I'm not even sure if that's the best tool. I'm open to using anything commonly available on a Linux system.

In perl, I did it like this ... it works, but it's dirty and has a trailing comma. Was hoping for something simpler:

$ perl -ne 'if (/^(\w)\R/) {print "$1,";} else {print "\n";}' test
A,B,C,D,
E,F,G,H,
I,J,K,L,

Matt Jacob · Answer 1 · 2018-02-23T05:11:56.783

8

Set the input record separator to paragraph mode (-00) and then split each record on any remaining whitespace:

$ perl -00 -ne 'print join("," => split), "\n"' test

Add -l to enable automatic newlines (but make sure it comes before -00, because we want $\ to be set to the value of $/ before modification):

$ perl -l -00 -ne 'print join("," => split)' test

Add -a to enable autosplit mode and implicitly split to @F:

$ perl -l -00 -ane 'print join("," => @F)' test

Swap out -n for -p for automatic printing:

$ perl -l -00 -ape '$_ = join("," => @F)' test

edited Feb 23 '18 at 05:11

answered Feb 22 '18 at 23:08

Matt Jacob

6,503
2
24
27

2

I came up with `perl -00nE 'chomp; s/\s+/,/g; say' file` to prevent double spacing the output. – glenn jackman Feb 22 '18 at 23:21
1

@GillesQuenot Because it will set the output record separator to the same value as the input record separator if it's omitted, and we don't want that. So, we have to set it first, before modifying `$/`. – Matt Jacob Feb 22 '18 at 23:52
@ikegami yup, it would work `perl -l -00 -ane 'print join ",", @F'` – Sundeep Feb 23 '18 at 03:55
Adding `-F$"\n"` would support lines with space. – ikegami Feb 23 '18 at 05:28

glenn jackman · Answer 2 · 2018-02-23T04:28:04.183

3

You could use

awk 'BEGIN {RS=""; FS="\n"; ORS="\n"; OFS=","} {$1=$1} 1' file

I see the gawk manual says this:

If RS is set to the null string, then records are separated by blank lines. When RS is set to the null string, the newline character always acts as a field separator, in addition to whatever value FS may have.

So we don't actually need to specify FS to get the desired output:

awk 'BEGIN {RS=""; ORS="\n"; OFS=","} {$1=$1} 1' file

edited Feb 23 '18 at 04:28

answered Feb 22 '18 at 23:24

glenn jackman

238,783
38
220
352

3

You also don't need to set ORS since `\n` is the default - `awk -v RS= -v OFS=, '{$1=$1}1'` – Ed Morton Feb 23 '18 at 12:09

score 2 · Answer 3 · answered Feb 22 '18 at 22:47

2

xargs could do it,

$ xargs -n4 < file | tr ' ' ','
A,B,C,D
E,F,G,H
I,J,K,L

answered Feb 22 '18 at 22:47

iamauser

11,119
5
34
52

Socowi · Answer 4 · 2018-02-22T22:39:43.333

Replacing newlines with sed is a bit complicated (see this question). It is easier to use tr for the newlines. The rest can be done by sed.

The following command assumes that yourFile does not contain any ,.

tr '\n' , < yourFile | sed 's/,*$/\n/;s/,,/\n/g'

The tr part converts all newlines to ,. The resulting string will have no newlines.
s/,*$/\n/ removes trailing commas and appends a newline (text files usually end with a newline).
s/,,/\n/g replaces ,, by a newline. Two consecutive commas appear only where your original file contained two consecutive newlines, that is where the sections are separated by an empty line.

Data transformation using sed

4 Answers4