Use variables in awk and/or sed

Question

In bash script from the output below, I need to print the lines between "Device #0" and "Device #1", but as all that is part of a bigger script I should use variables for start/stop lines.

----------------------------------------------------------------------
Physical Device information
----------------------------------------------------------------------
      Device #0
         Device is a Hard drive
         State                              : Online
         Block Size                         : 512 Bytes
         Supported                          : Yes
         Programmed Max Speed               : SATA 6.0 Gb/s
         Transfer Speed                     : SATA 6.0 Gb/s
         Reported Channel,Device(T:L)       : 0,0(0:0)
         Reported Location                  : Connector 0, Device 0
         Vendor                             : ATA
         Model                              :

         Firmware                           : 003Q
         Serial number                      : S2HTNX0H418779
         World-wide name                    : 5002538C402805A4
         Reserved Size                      : 265496 KB
         Used Size                          : 897129 MB
         Unused Size                        : 18327 MB
         Total Size                         : 915715 MB
         Write Cache                        : Enabled (write-back)
         FRU                                : None
         S.M.A.R.T.                         : No
         S.M.A.R.T. warnings                : 0
         Power State                        : Full rpm
         Supported Power States             : Full power,Powered off
         SSD                                : Yes
         Temperature                        : 39 C/ 102 F
         NCQ status                         : Enabled
      ----------------------------------------------------------------
      Device Phy Information
      ----------------------------------------------------------------
         Phy #0
            PHY Identifier                  : 0
            SAS Address                     : 30000D1701801803
            Attached PHY Identifier         : 3
            Attached SAS Address            : 50000D1701801800
      ----------------------------------------------------------------
      Runtime Error Counters
      ----------------------------------------------------------------
         Hardware Error Count               : 0
         Medium Error Count                 : 0
         Parity Error Count                 : 0
         Link Failure Count                 : 0
         Aborted Command Count              : 0
         SMART Warning Count                : 0


Model, SSD

----------------------------------------------------------------------
Physical Device information
----------------------------------------------------------------------
      Device #1
         Device is a Hard drive
         State                              : Online
         Block Size                         : 512 Bytes
         Supported                          : Yes
         Programmed Max Speed               : SATA 6.0 Gb/s
         Transfer Speed                     : SATA 6.0 Gb/s
         Reported Channel,Device(T:L)       : 0,0(0:0)
         Reported Location                  : Connector 0, Device 0
         Vendor                             : ATA
         Model                              :

         Firmware                           : 003Q
         Serial number                      : S2HTNX0H418779
         World-wide name                    : 5002538C402805A4
         Reserved Size                      : 265496 KB
         Used Size                          : 897129 MB
         Unused Size                        : 18327 MB
         Total Size                         : 915715 MB
         Write Cache                        : Enabled (write-back)
         FRU                                : None
         S.M.A.R.T.                         : No
         S.M.A.R.T. warnings                : 0
         Power State                        : Full rpm
         Supported Power States             : Full power,Powered off
         SSD                                : Yes
         Temperature                        : 39 C/ 102 F
         NCQ status                         : Enabled
      ----------------------------------------------------------------
      Device Phy Information
      ----------------------------------------------------------------
         Phy #0
            PHY Identifier                  : 0
            SAS Address                     : 30000D1701801803
            Attached PHY Identifier         : 3
            Attached SAS Address            : 50000D1701801800
      ----------------------------------------------------------------
      Runtime Error Counters
      ----------------------------------------------------------------
         Hardware Error Count               : 0
         Medium Error Count                 : 0
         Parity Error Count                 : 0
         Link Failure Count                 : 0
         Aborted Command Count              : 0
         SMART Warning Count                : 0


Model, SSD

----------------------------------------------------------------------
Physical Device information
----------------------------------------------------------------------
      Device #2
         Device is a Hard drive
         State                              : Online
         Block Size                         : 512 Bytes
         Supported                          : Yes
         Programmed Max Speed               : SATA 6.0 Gb/s
         Transfer Speed                     : SATA 6.0 Gb/s
         Reported Channel,Device(T:L)       : 0,0(0:0)
         Reported Location                  : Connector 0, Device 0
         Vendor                             : ATA
         Model                              :

         Firmware                           : 003Q
         Serial number                      : S2HTNX0H418779
         World-wide name                    : 5002538C402805A4
         Reserved Size                      : 265496 KB
         Used Size                          : 897129 MB
         Unused Size                        : 18327 MB
         Total Size                         : 915715 MB
         Write Cache                        : Enabled (write-back)
         FRU                                : None
         S.M.A.R.T.                         : No
         S.M.A.R.T. warnings                : 0
         Power State                        : Full rpm
         Supported Power States             : Full power,Powered off
         SSD                                : Yes
         Temperature                        : 39 C/ 102 F
         NCQ status                         : Enabled
      ----------------------------------------------------------------
      Device Phy Information
      ----------------------------------------------------------------
         Phy #0
            PHY Identifier                  : 0
            SAS Address                     : 30000D1701801803
            Attached PHY Identifier         : 3
            Attached SAS Address            : 50000D1701801800
      ----------------------------------------------------------------
      Runtime Error Counters
      ----------------------------------------------------------------
         Hardware Error Count               : 0
         Medium Error Count                 : 0
         Parity Error Count                 : 0
         Link Failure Count                 : 0
         Aborted Command Count              : 0
         SMART Warning Count                : 0


Model, SSD

In this case the output for Device #0 to Device #2 is the same, but it doesn't really matter for the test. So trying with cat arcconf | awk '/Device #0/,/Device #1/' where the output above is stored in a file called arcconf works. But trying to use variables instead of 0 and 1 doesn't work at all:

MIN_INDEX=0
INDEX=1
cat arcconf | awk '/Device #"$MIN_INDEX"/,/Device #"$INDEX"/'
cat arcconf | sed -n -e "/Device #"$INDEX_MIN"$/,/Device #"$INDEX"$/{ /Device #"$INDEX_MIN"$/d; /Device #"$INDEX"$/d; p; }"

It doesn't display anything. Could you please help.

Also as I am going to use the output from Device to Device lines several times, is it possible to store it in some new variable which I should use in the future?

Thanks, Valentina

score 0 · Answer 1 · answered Nov 02 '16 at 06:31

0

Bash variables are not expanded within single quotes, that's why the first command doesn't work. Replace single quotes with double quotes:

cat arcconf | awk "/Device #$MIN_INDEX/,/Device #$INDEX/"

The second command should work, but it's unnecessarily complicated. You don't need to drop out of the double-quoted string for the sake of the variables, this will work fine:

cat arcconf | sed -n -e "/Device #$INDEX_MIN$/,/Device #$INDEX$/{ /Device #$INDEX_MIN$/d; /Device #$INDEX$/d; p; }"

In fact it's better this way, as now the variables are within a double-quoted string, which is a good habit, as unquoted variables containing spaces would cause problems.

answered Nov 02 '16 at 06:31

janos

120,954
29
226
236

Thank you Janos. Seems like it is working now. How about adding this output in another variable? With OUTPUT=`cat arcconf | awk "/Device #$MIN_INDEX/,/Device #$INDEX/"` and then printing the OUTPUT variable, it shows everything on one line. Is there a way to write it in a variable just the way it is shown on the screen(I guess maybe arrays should be used because of the new lines)? – maimun4itu Nov 02 '16 at 06:41
That's an entirely different question... but ok: I guess you are doing `echo $OUTPUT` when the output is on one line. To display on multiple lines, double-quote the variable in the `echo`, like this: `echo "$OUTPUT"` – janos Nov 02 '16 at 21:45

score 0 · Answer 2 · answered Nov 02 '16 at 06:52

You can send variables to awk via -v var=val:

awk \
  -v start="Device #$MIN_INDEX" \
  -v end="Device #$MAX_INDEX" \
  '$0 ~ end   { p=0 }
   $0 ~ start { p=1 }
   p' arcconf

Simply by moving around p; you can whether or not to include the start and end line:

$0 ~ end { p=0 }; p; $0 ~ start { p=1 } # Will not include start nor end
$0 ~ end { p=0 }; $0 ~ start { p=1 }; p # Will include start and end
$0 ~ start { p=1 }; p; $0 ~ end { p=0 } # Will include start but not end
$0 ~ end { p=0 }; p; $0 ~ start { p=1 } # Will include end but not start

VIPIN KUMAR · Answer 3 · 2016-11-02T11:43:39.060

You can try below sed command -

#MIN_INDEX=0
#INDEX=1
#sed -n "/Device\ #$MIN_INDEX/,/Device\ #$INDEX/p" kk.txt

And to set the output to a variable -

#sed -n "/Device\ #$MIN_INDEX/,/Device\ #$INDEX/w output.txt" kk.txt
#var=`cat output.txt`
#echo $var

Explanation

-n to remove duplicate when pattern match.

w is to write the output to file output.txt

p is to print. We need to use escape character \ to search space.

Use variables in awk and/or sed

3 Answers3