printing files in linux using awk command for today's date with some given pattern

Question

I am new to linux and trying to learn awk command. I have a directory in which I am trying to list some particular files for today's date only as they are generated everyday.

When I do ls -lrt in directory it shows ouput like below -

-rw-rw----   1 mnp8u      mnp8u      1774165 Aug  2 12:39 SP055Process.log
-rw-rw----   1 mnp8u      mnp8u      3207535 Aug  2 12:39 SP120Process.log
-rw-rw----   1 mnp8u      mnp8u      2127814 Aug  2 12:39 SP115Process.log
-rw-rw----   1 mnp8u      mnp8u      3818378 Aug  2 12:39 SP003Process.log
-rw-rw----   1 mnp8u      mnp8u      1746325 Aug  2 12:39 SP051Process.log
-rw-rw----   1 mnp8u      mnp8u      1962051 Aug  2 12:40 SP111Process.log
-rw-rw----   1 mnp8u      mnp8u      2183470 Aug  2 12:40 mnpcOutboundAdapter.log
-rw-rw----   1 mnp8u      mnp8u      6982717 Aug  2 12:40 performance_outbound_adapter.log
-rw-rw----   1 mnp8u      mnp8u      4016589 Aug  2 12:40 SP100Process.log
-rw-rw----   1 mnp8u      mnp8u      3891733 Aug  2 12:40 SP110Process.log
-rw-rw----   1 mnp8u      mnp8u      3128098 Aug  2 12:40 SP107Process.log
-rw-rw----   1 mnp8u      mnp8u      3776865 Aug  2 12:40 SP114Process.log
-rw-rw----   1 mnp8u      mnp8u       661324 Aug  2 12:40 SP201Process.log
-rw-rw----   1 mnp8u      mnp8u      4837389 Aug  2 12:40 SP122Process.log
-rw-rw----   1 mnp8u      mnp8u      4160117 Aug  2 12:40 SP117Process.log
-rw-rw----   1 mnp8u      mnp8u      1329318 Aug  2 12:40 performance_cm_workflow.log
-rw-rw----   1 mnp8u      mnp8u      14672012 Aug  2 12:40 performance_service_provider.log.1
-rw-rw----   1 mnp8u      mnp8u      4362403 Aug  2 12:40 SP304Process.log
-rw-rw----   1 mnp8u      mnp8u      3868420 Aug  2 12:40 SP301Process.log
-rw-rw----   1 mnp8u      mnp8u      3039108 Aug  2 12:40 controllerProcess.log
-rw-rw----   1 mnp8u      mnp8u      2390825 Aug  2 12:40 performance_service_provider.log
-rw-rw----   1 mnp8u      mnp8u      77257784 Aug  2 12:40 cmserver.log
-rw-rw----   1 mnp8u      mnp8u      1210801 Aug  2 12:40 SP105Process.log
-rw-rw----   1 mnp8u      mnp8u      3541792 Aug  2 12:40 receiverProcess.log
-rw-rw----   1 mnp8u      mnp8u      3118069 Aug  2 12:40 SP109Process.log
-rw-rw----   1 mnp8u      mnp8u      2619817 Aug  2 12:40 x400Audit.log
-rw-rw----   1 mnp8u      mnp8u      8138062 Aug  2 12:40 x400.log
-rw-rw----   1 mnp8u      mnp8u      5328634 Aug  2 12:40 controllerProcess.log.34
-rw-rw----   1 mnp8u      mnp8u      4830514 Aug  2 12:40 mrdbSenderProcess.log
-rw-rw----   1 mnp8u      mnp8u      6844952 Aug  2 12:40 performance_mrdb.log
-rw-rw----   1 mnp8u      mnp8u       125466 Aug  2 12:40 SP113Process.log
-rw-rw----   1 mnp8u      mnp8u      4284622 Aug  2 12:40 cm_http_server.log

There are lots of files that i have skipped in above code.

I am trying to print 3 particular files (performance_outbound_adapter.log, mnpcOutboundAdapter.log, problems.log) for today's date only (2nd August, 2021) using awk command as shown below -

[deuxvi1e:/pkg/vdc8u/logs/mnp8u/cm][mnp8u]$ ls -lrt|awk '$7==2 && /performance_outbound_adapter.log/{print $0}
        $7==2 && /mnpcOutboundAdapter.log/{print $0}
>       $7==2 && /mnpcOutboundAdapter.log/{print $0}2 && /problems.log/{print $0}'
>       $7==2 && /problems.log/{print $0}'
-rw-rw----   1 mnp8u      mnp8u      5120217 Aug  2 04:43 mnpcOutboundAdapter.log.2
-rw-rw----   1 mnp8u      mnp8u      10242528 Aug  2 08:26 problems.log.1
-rw-rw----   1 mnp8u      mnp8u      5120385 Aug  2 10:25 mnpcOutboundAdapter.log.1
-rw-rw----   1 mnp8u      mnp8u      22237754 Aug  2 12:40 problems.log.2
-rw-rw----   1 mnp8u      mnp8u      1185693 Aug  2 12:42 problems.log
-rw-rw----   1 mnp8u      mnp8u      8029066 Aug  2 12:42 mnpcOutboundAdapter.log.4
-rw-rw----   1 mnp8u      mnp8u      6984351 Aug  2 12:42 performance_outbound_adapter.log
-rw-rw----   1 mnp8u      mnp8u      2195604 Aug  2 12:45 mnpcOutboundAdapter.log
[deuxvi1e:/pkg/vdc8u/logs/mnp8u/cm][mnp8u]$

Above I am using $7==2 as today's date is 2nd August for every pattern.

Is there any way by which I can use date command so that I don't have to put date value like 2, 3 every day in condition $7==date ?

Someone please help me here.

Answer 1

I am trying to print 3 particular files (performance_outbound_adapter.log, mnpcOutboundAdapter.log, problems.log) for today's date only (2nd August, 2021)

Kindly try:

find . -maxdepth 1 -mindepth 1 -type f -mtime -1 '(' -name 'performance_outbound_adapter.log*' -o -name 'mnpcOutboundAdapter.log*' -o -name 'problems.log*' ')'

For "since 00:00:00 today" you could calculate the number of minutes and use -mmin:

min=$(( ( $(date +%s) - $(date -d 00:00:00 +%s) ) / 60 ))
find .... -mmin -"$min" ....

If you want to do some parsing in awk, use find with -printf, at best use zero separated streams. Use timestamps in seconds since epoch. A crude example:

find . <someargs...> -printf "%Ts %p\n" |
awk -v date=$(date -d '00:00:00' +%s) '$1 > date{ print $2 }'

Answer 2

Since Unix doesn't store file creation time, the best you can do is look at the files last modification time assuming it hasn't been modified since it was created. If you NEED creation time then you need to implement something yourself to store the time when the file is created so you can access it later.

Below I'm using solutions that will work even if you used * or similar instead of a specific list of file names but applied to your specific list of file names.

You could use stat if you only have files in your directory and/or don't want to visit sub-directories and your stat+awk support NUL-terminated input:

stat --printf '%y %n\0' performance_outbound_adapter.log* mnpcOutboundAdapter.log* problems.log* |
awk -v RS='\0' -v date=$(date +'%F') '$1 == date{ sub(/.{35}/,""); print }'

otherwise if your file names don't contain newlines (but can contain any other chars):

stat --format '%y %n' ... |
awk -v date=$(date +'%F') '$1 == date{ sub(/.{35}/,""); print }'

Otherwise if you wanted to visit sub-directories and/or exclude directories from your output or have any other better control of which files/directories are examined, then use find instead of stat:

If your find+awk support NUL terminated strings:

find . '(' -name 'performance_outbound_adapter.log*' -o -name 'mnpcOutboundAdapter.log*' -o -name 'problems.log*' ')' -printf '%TF %p\0' |
awk -v RS='\0' -v date=$(date +'%F') '$1 == date{ sub(/.{11}/,""); print }'

otherwise (again, won't support file names that contain newlines):

find . ... -printf '%TF %p\n' |
awk -v date=$(date +'%F') '$1 == date{ sub(/.{11}/,""); print }'

It may be worth mentioning that you could have used sed instead of awk in the above, e.g.:

stat --format '%y %n' * |
sed -n "s/^$(date +'%F').\{26\}//p"

but it gets a bit trickier when trying to handle NUL-terminated input as that requires GNU sed for -z and generating newline-terminated output from that isn't intuitive whereas it's trivial with awk if you want it (just leave ORS as \n or set it to \0 as you prefer).