Insert or update a row in a text file

Question

I want to write a value to a text file with a timestamp. The data/value gets updated every few minutes, but I only want to record the last value and its date. The next day I want to start again with a new line but keeping the previous value. The output will then be charted showing daily values.

I can do this with an if, then, else loop, but I imagine there's a more elegant awk, sed, 'something' solution.

This will be run by a cron job.

Day 1. Every 5 minutes insert a row with $date, $value

During the same day keep updating that row

Day 2. Insert a new row and repeat

Desired output

Day 1 10am

2021-08-17, 5.20

Day 1 11pm

2021-08-17, 12.10

Day 2 10am

2021-08-17, 12.10
2021-08-18, 4.90

Day 2 11pm

2021-08-17, 12.10
2021-08-18, 13.10

_I can do this with an if, then, else loop_ There is no _if-then-else-loop_. Loops in bash are done using _for_, _while_ or _select_. — user1934428, Aug 18 '21 at 06:30

markp-fuso · Accepted Answer · 2021-08-18T00:11:40.377

One idea would be to skip/ignore/delete the line with today's date (if it exists) and then append a new line with today's date.

Sample data file:

$ cat date.log
2021-08-14, 23.1
2021-08-15, 17.3
2021-08-16, 9.3

$ today=$(date '+%Y-%m-%d')
$ echo $today
2021-08-17

$ newvalue=13.2

One sed idea to implement this logic:

$ sed -i -n -e "/^${today}, .*$/"'!p' -e '$a'"${today}, ${newvalue}" date.log

Where:

-i -n -e - -inplace update of source file, -n suppress automatic printing of pattern space, -e designate a piece of script
"/^${today}, .*$/" - search for pattern that matches (start of line) + ${today} + , + rest of line; need to use double quotes so that ${today} is replaced with it's actual value
'!p' - reverse pattern search and print line (ie, print everything but line that matches ^${today}, .*$); need to use single quotes since !p in double quotes will be replaced with the last history command that started with the letter p
-e '$a' - another piece of script that finds end of file ($) and appends the following string; must use single quotes so that bash doesn't try to replace the literal $a with the contents of variable a
"${today}, ${newvalue}" - string to append to end of file

If we preface our sed call with set -xv (enable debug mode) we see the following is printed at the console:

+ sed -i -n -e '/^2021-08-17, .*$/!p' -e '$a2021-08-17, 13.2' date.log

And the contents of our file:

$ cat date.log
2021-08-14, 23.1
2021-08-15, 17.3
2021-08-16, 9.3
2021-08-17, 13.2              # new line appended to file

A couple more runs (after issuing set +xv to disable debug mode):

$ newvalue=32.7
$ sed -i -n -e "/^${today}, .*$/"'!p' -e '$a'"${today}, ${newvalue}" date.log
$ cat date.log
2021-08-14, 23.1
2021-08-15, 17.3
2021-08-16, 9.3
2021-08-17, 32.7              # updated (really deleted and appended)

$ newvalue=73.xxx
$ sed -i -n -e "/^${today}, .*$/"'!p' -e '$a'"${today}, ${newvalue}" date.log
$ cat date.log
2021-08-14, 23.1
2021-08-15, 17.3
2021-08-16, 9.3
2021-08-17, 73.xxx            # updated (really deleted and appended)

$ today='2021-09-23'
$ newvalue=7.25
$ sed -i -n -e "/^${today}, .*$/"'!p' -e '$a'"${today}, ${newvalue}" date.log
$ cat date.log
2021-08-14, 23.1
2021-08-15, 17.3
2021-08-16, 9.3
2021-08-17, 73.xxx
2021-09-23, 7.25              # new line appended to file

Thanks @markp-fuso - that's just what I was after. And thank you for the detailed and clear explanation - much appreciated — maxillis, Aug 18 '21 at 05:29

score 0 · Answer 2 · answered Aug 17 '21 at 22:36

Use two different cronjob entries. One that runs at the start of each day that adds a new row to the file with ed or whatever:

# Something that sets date and value here
printf "%s\n" '$a' "$date, $value" . w | ed -s /your/file.txt

And another that runs every 5 minutes and replaces the current last line of the file with a new value:

# Something that sets date and value here
printf "%s\n" '$c' "$date, $value" . w | ed -s /your/file.txt

score 0 · Answer 3 · answered Aug 18 '21 at 05:33

Using -i inplace capable version of GNU awk:

$ gawk -i inplace -v d=2021-08-17 -v v=12.20 '    # new data in parameters
@load "filefuncs"                                 # for stat to detect missing file
BEGIN {
    if(ARGC==1)                                   # if no file given
        exit 1                                    # exit with an error
    FS=OFS=", "                                   # set the field separators
    # d=strftime("%F")                            # no need parameterize the date
    if(stat(ARGV[1],fdata)<0) {                   # if given file does not exist
        print d,v > ARGV[1]                       # create it
        exit                                      # and exit
    }
}
{
    if(NR>1)                                  
        print pd,pv                               # print previous data record
    pd=$1                                         # store current for next round
    pv=$2
}
ENDFILE {                                         # in the end
    if(pd!=d)                                     # if dates differ
        print pd,pv                               # print previous
    print d,v                                     # print given data
    exit                                          # to skip over following files
}' file

Insert or update a row in a text file

3 Answers3