sed: How to replace variable line if found or append to end of file if not found?

Question

So my question is very much based off of a related question, but has one key difference.

In the answer given of:

sed '/^FOOBAR=/{h;s/=.*/=newvalue/};${x;/^$/{s//FOOBAR=newvalue/;H};x}' infile

I've made a variation that works for making FOOBAR a variable (e.g. name of a program and then putting it in a for loop to make a list of programs and their installation status). This is for a rather involved project I'm working on to make the installation of Drawile for Raspberry Pi easy with some new features, but that's a different can of worms I'm not going to get into.

sed -i '/^'"$FOOBAR"'=/{h;s/=.*/=newvalue/};${x;/^$/{s//'"FOOBAR"'=newvalue/;H};x}' infile

I would like newvalue to be a $variable which can change in the script. I just can't figure out how to make that work, simply using ' and " doesn't do it right. I've spent hours browsing and trying to read tutorials, but I just don't understand enough and have dived very deep into the inner workings of sed. Most of the time, this variable will be something simple like: yes, no, or error. However, I might also use it to represent folder and file paths... so I'm worried about / substitutions as well here. Even if it's not a full answer, I'm happy to figure it out myself as long as I can get pointed in the direction that I need to go to understand this better.

---

Edit: There isn't much to post given that there's something fundamental I'm sure I'm missing, I've just tried variations of putting quotes around newvalue. Such as follows...

sed -i '/^FOOBAR=/{h;s/=.*/=$newvalue/};${x;/^$/{s//FOOBAR=$newvalue/;H};x}' infile

Output: FOOBAR = $newvalue
This is incorrect. Should output the stored value for newvalue!

sed -i '/^FOOBAR=/{h;s/=.*/="$newvalue"/};${x;/^$/{s//FOOBAR="$newvalue"/;H};x}' infile

Output: FOOBAR = "$newvalue"
This is incorrect. Should output the stored value for newvalue!

sed -i '/^FOOBAR=/{h;s/=.*/='"$newvalue"'/};${x;/^$/{s//FOOBAR='"$newvalue"'/;H};x}' infile

Output: sed: -e expression #1, char 30: unknown option to `s'

If newvalue is equal to Yes, then I'd want the output to be FOOBAR = Yes or possibly a folder path.

Answer 1

Let's look at the basic form of sed.

Usage: sed [OPTION]... {script-only-if-no-other-script} [input-file]...

Here's the answer I arrived at.

sed -i '\!^'"$FOOBAR"'=!{h;s!=.*!='"$newvalue"'!};${x;\!^$!{s!!'"$FOOBAR"'='"$newvalue"'!;H};x}' /home/pi/Public/test.txt

Explanation from left to right:

• -i is used to edit the file in place, meaning the original file is edited with no backup.
• ' is used to start the expression script that we'll use to achieve our goal.
• ! must be used in place of the typical / delimiter, since we're expecting input that may use the / in a folder or file path. The first exclamation mark is escaped to make sure it is interpreted correctly
• This first portion looks for an existing copy of whatever $FOOBAR is, such as apache2 for our program installation status. The bullet points that follow are specific to this portion of our sed query.
  • ^ is a Regex expression meaning that it will match the start position of a line. Meaning that it would match $FOOBAR at the beginning of a line and not any of the following: $FOOBAR, THIS IS $FOOBAR, etc.
  • '" and "' surrounds $FOOBAR, as it's needed to make sure the script interprets the variable correctly. Without it, it'd end up printing $FOOBAR into the document instead of the value contained by the variable $FOOBAR. The equal sign that follows is simply part of the expected portion of text we're looking for.
  • The { is used to group commands.
  • The h command copies the pattern buffer into the hold buffer to be used later. This is a single line buffer, so it can't hold more than one line at a time.
  • The ; command is used to combine several sed commands on one line. Unfortunately, it seems rather poorly documented. You can read more here http://www.grymoire.com/Unix/Sed.html#uh-61
  • The s signifies part of a new pattern to be used for substitution.
  • .* is a regex expression meaning in this case, to look for an equals sign followed by any number of characters before a new line commences. It will replace this with an equals sign followed by $newvalue, such as = yes or = no. The variable $newvalue is surrounded by '" and "' for the same reasons $FOOBAR has them. We then put in the last delimiter and close the grouped command.
• This is now the second portion of our sed command. The bullet points that follow are specific to this portion of our sed query. Repeated portions will be skipped.
  • The $ looks for the the end of the last line in the file. We then begin a new grouped command series.
  • The x will exchange the pattern space with the hold buffer. This is the part where I get a bit fuzzy myself, but it sounds like to me that sed is modifying the entire file at this point, as noted here. I attempt to explain this more in the last bullet point. Read more here - http://www.grymoire.com/Unix/Sed.html#uh-53
  • The ^$ means we're looking for a paragraph return that is both the start and end of the line, with no characters of any kind on this line.
  • We then move on to (s)ubstitute that no character line with the double delimiter exclamation mark and (s)ubstitute that value with the variables $FOOBAR and $newvalue.
  • The H is the same as the lower case h in that it's a hold buffer, but one intended for multiple lines. We close that grouping and substitute it again with x. What this basically does is first insert our desired line at the top of the document. Sed moves through the document line by line, taking the line that follows below it and moving it above the line we're inserting. Since sed works like a "pipe", being a stream editor, it will do this until it reaches the end of the document and stops processing changes.
• And the last of it is the file to be modified: /home/pi/Public/test.txt.

I think I managed to explain everything, feedback is welcomed.