Replace occurrences of a string in file with different bytes of entropy each

Question

On Linux, I need to programmatically replace placeholder strings such as <SECRET> in a .env file like this:

KEY=<SECRET>
ANOTHER_VARIABLE=another-value
# here's a comment
PASSWORD=<SECRET>

The caveat is, that each occurrence of this placeholder must be replaced with a different instantiation of Base64 encoded randomness - e.g. from OpenSSL, since it's readily available on many Linuxes.

Reading this answer, I tried this with GNU sed 4.8:

sed -i '0,/<SECRET>/ s__'$(openssl rand -base64 42)'_' .env

(In the substitution part the alternative delimiter _ was chosen, because the Base64 encoded bytes can contain / or + characters and would otherwise clash when inadvertently used as delimiters.)

This works for single replacements, one call at a time.

But sed's return code is always 0, even when all occurrences of the regex have been consumed and replaced...

Question: Is there a way to make sed return a non-zero code when placeholders have been exhausted?

(If this can't be done with sed, I'm happy for any solution with awk or similar.)

@Quasímodo i've tried this already, but this also didn't work - return codes were still 0. — rypel, Feb 29 '20 at 23:21
i've ended up grep'ing for the placeholder in a while loop before doing the substitution with sed, as @Quasímodo suggested. — rypel, Feb 29 '20 at 23:59

score 2 · Accepted Answer · answered Feb 29 '20 at 23:34

2

Instead of sed, you could use grep:

grep '<SECRET>' .env

From man grep:

EXIT STATUS

Normally the exit status is 0 if a line is selected, 1 if no lines were selected, and 2 if an error occurred. However, if the -q or --quiet or --silent is used and a line is selected, the exit status is 0 even if an error occurred.

If the return value is 0, then apply your sed command to perform the substitution.

answered Feb 29 '20 at 23:34

Quasímodo

3,812
14
25

thanks, this did the trick: simply wrapping the sed substitution in a while condition with grep: `while grep -q '' test; do sed -i '0,// s__'$(openssl rand -base64 42)'_' test; done` – rypel Feb 29 '20 at 23:55

Ed Morton · Answer 2 · 2020-03-01T03:18:45.007

$ awk '
    BEGIN { cmd = "openssl rand -base64 42" }
    match($0,/<SECRET>/) {
        val = ( (cmd | getline line) > 0 ? line : "N/A" )
        $0 = substr($0,1,RSTART-1) val substr($0,RSTART+RLENGTH)
        close(cmd)
    }
1' file
KEY=hDc3Bw4J9+TUbbhw4cNKda+mDHVRGGrAWUU6LX7aYZnWwDZWyqrmzi3z
ANOTHER_VARIABLE=another-value
# here's a comment
PASSWORD=LzGaB43Mm5mF6tsJnwOLqgeeoTwajH9FNLty9yD22QovadhwWKpr7AP6

or if <SECRET> can occur multiple times on 1 line then:

$ awk '
    BEGIN { cmd = "openssl rand -base64 42" }
    {
        while ( match($0,/<SECRET>/) ) {
            val = ( (cmd | getline line) > 0 ? line : "N/A" )
            $0 = substr($0,1,RSTART-1) val substr($0,RSTART+RLENGTH)
            close(cmd)
        }
    }
1' file
KEY=t3/bimGkyPQcjOL6ubV6QmRnPjrvtg4+HvvkuJSnuYCYc+BzOmAWKRV6
ANOTHER_VARIABLE=another-value
# here's a comment
PASSWORD=TKvDbMphOhmhL0/NSQEE2Gs9sFr3Cwt9o3CbOoe9FZPjAR/+m6i4QjcR

Replace occurrences of a string in file with different bytes of entropy each

2 Answers2