78

I have a Linux program which can write information to stdout and stderr.

I have a shell script which redirects that output to a file in /var/log. (Via >> and 2>&1.)

Is there a way to make that log file rotate? (max size, then switch to a different file, keep only a limited number of files)

I've seen a few answers which talk about the logrotate program, which sounds good, but they also seem to be focused on programs which are generating log files internally and handle HUP signals. Is there a way to make this work with a basic output redirection script?

Miral
  • 1,269

11 Answers11

66

As an alternative, you could pipe the output through tools designed with the primary purpose of maintaining size-capped, automatically rotated, log file sets, such as:

Tools to then process multilog-format log file sets include, amongst others:

Further reading

jonathan
  • 105
JdeBP
  • 27,556
  • 1
  • 77
  • 106
20

the rotatelogs tool shipped with apache (in the bin dir) (see docs) takes input from stdin and rotates the log after some specific amount of time

BertNase
  • 301
15

I had similar problem and had initially discard logrotate but it turned out logrotate can actually do this well, the key directive is "copytruncate". For some reason that term didn't come up on any of the googling I did, so I am adding this answer to clarify exactly how to use it for this case.

The trick is this only works if the redirect is done with ">>" (append) instead of ">" (create).

Config File (truncate.cfg):

/tmp/temp.log {
    size 10M
    copytruncate
    rotate 4
    maxage 100
}

Test Program (never gives up file). You can watch it filling disk and though deleting logfile will appear to work it will not actually free up any space on the disk:

cat /dev/urandom >> /tmp/temp.log

Running log rotate:

logrotate truncate.cfg
Sam Hendley
  • 251
  • 2
  • 3
14

If you can have it go to one of the standard log streams (syslog, daemon, cron, user, security, mail, etc.) you can use the logger command and pipe to it instead.

echo "Hello." | logger -p daemon.info

Otherwise, you may be better off piping your logged content to a custom program or script to handle it, or look at setting up the logrotate configuration.

EDIT: JdeBP's answer seems to have what you may be looking for.

Lara Dougan
  • 2,296
5

I like multilog for my use case, but my use case is so trivial/simple that it is not laid out very simply in the docs/examples I found. Here is a simple multilog rotate example:

mkdir /tmp/myapp
./myapp | multilog t s10000 n5 '!tai64nlocal' /tmp/myapp 2>&1

Some notes:

  • this dumps logs into that /tmp/myapp/ directory
  • the s10000 represents 10,000 bytes*
  • the n5 represents 5 files.* The 'current' log counts as one of the files, so this includes 4 older logs + 'current'
  • this is based on, adapted from the examples provided by François Beausoleil at: http://blog.teksol.info/pages/daemontools/best-practices
  • I don't understand many of the options - I refer you to the various documentation to extend this...
  • The docs warn that: "Note that running processor may block any program feeding input to multilog." where 'processor' is the '!tai64nlocal' portion of the command

*For many applications, these are poor choices for long term use. They do allow you to observe the behavior of filling and rotating the logs more quickly than large logs do.

Finally, don't forget to nohup if required! With nohup, you do not need the 2>&1 (s=10e6 and n=30 here):

mkdir -p /tmp/myapp
nohup ./myapp | multilog t s10000000 n30 '!tai64nlocal' /tmp/myapp &

That command should get you started.

sage
  • 1,227
3

So is there a way to make logrotate work with an ongoing process's redirected stdout?

Yes! Check out the "copytruncate" directive offered by logrotate. Specifying that instructs logrotate to handle this very situation: a simple program that keeps its log file open indefinitely.

One caveat may or may not be a problem in your situation:

Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost.

Anecdotally, I've seen some "real world" log sources that do encourage users to apply this directive. There's some discussion of this option here.

Community
  • 1
natevw
  • 840
3

Use split, it's part of coreutils. It can take stdin and split it into chunks (based on chunk size, or number of lines, etc.).

Example:

app | split --bytes 1G - /var/logs/put-prefix-here

Note dash (-) instructs "split" to use stdin instead of file.

Nazar
  • 131
1

I just wanted to add to Sam Hendley's comment above:

The trick is this only works if the redirect is done with >> (append) instead of > (create).

I ran into the same problem where the original file just keeps growing if you use > (create) but if you use >> (append) Logrotate copytruncate works beautifully and as expected. The original file goes back down to zero bytes and the program continues writing.

Redirect STDOUT and STDERR to a rotating logfile:

  1. some-program.sh >> /tmp/output.txt 2>&1 &

  2. Create a logrotate config file under /etc/logrotate.d called whatever, output_roll in my case.

    Sample config for my case:

    /tmp/output.txt {
    notifempty
    missingok
    size 1G
    copytruncate
    start 0
    rotate 15
    compress
}

  3. Setup your cron job inside the /etc/crontab file

    *  *  *  *  * root /usr/sbin/logrotate /etc/logrotate.d/output_roll

    This will check the file every minute. You can adjust to suit your needs.

  4. Start it up:

    $> service crond restart

  5. That's it

Note: I also had a problem with SELinux being set to SELINUX=enforcing so I set it to SELINUX=disabled.

kenorb
  • 26,615
user578558
  • 11
1

I wrote a logrotee this weekend. I probably wouldn't if I've read @JdeBP's great answer and multilog.

I focused on it being lightweight and being able to bzip2 its output chunks like:

verbosecommand | logrotee \
  --compress "bzip2 {}" --compress-suffix .bz2 \
  /var/log/verbosecommand.log

There's a lot of to be done and tested yet, though.

Victor Sergienko
  • 780
1

you can use such a simple bash scripts: ./my_app|./log_rotate.sh log.txt 5 mylogdir 10 1000

log named as log.txt ,rotate 5 times,stored in mylogdir, rotate 10 times,log rotate on 1000 lines, dir rotate on each start up of my_app

log_rotate.sh:

#!/bin/sh
logname=$1
lognumber=$2
dirname=$3
dirnumber=$4
line=$5
date
mkdir -p ${dirname}
cd ${dirname}
rm "data${dirnumber}" -rfv
for i in $(seq $((dirnumber - 1)) -1 0)
do
        mv "data$i" "data$(($i+1))" 2>/dev/null && echo mv "data$i" "data$(($i+1))"
done
mkdir -p data0
cd data0
dd of="${logname}" bs=2048 count=${line} 2>/dev/null
date
mv "${logname}" "${logname}.first" 2>/dev/null && echo mv "${logname}" "${logname}.first"
while true
do
        for i in $(seq $((lognumber - 1)) -1 1)
        do
                mv "${logname}.$i" "${logname}.$(($i+1))" 2>/dev/null && echo mv "${logname}.$i" "${logname}.$(($i+1))"
        done
        mv "${logname}" "${logname}.1" 2>/dev/null && echo mv "${logname}" "${logname}.1"
        dd of="${logname}" bs=2048 count=${line} 2>/dev/null
        date
done
Vick Sun
  • 11
0

As we were not fully satisfied with listed tools here, we wrote another one (sorry!) called log_proxy.

Main features:

  • usable as a pipe (myapp myapp_arg1 myapp_arg2 |log_proxy /log/myapp.log)
  • configurable log rotation suffix with stftime placeholders (for example: .%Y%m%d%H%M%S)
  • can limit the number of rotated files (and delete oldest)
  • can rotate files depending on their size (in bytes)
  • can rotate files depending on their age (in seconds)
  • does not need a specific log directory for a given app (you can have one directory with plenty of different log files from different apps)
  • several instances of the same app can log to the same file without issue (example: myapp arg1 |log_proxy --use-locks /log/myapp.log and myapp arg2 |log_proxy --use-locks /log/myapp.log can run at the same time)
  • implemented in C (fast and do not eat a lot of memory)
  • configurable with CLI options as well with env variables
  • usable as a wrapper to capture stdout and stderr (log_proxy_wrapper --stdout=/log/myapp.stdout --stderr=/log/myapp.stderr -- myapp myapp_arg1 myapp_arg2)
  • binary releases with no dependency, even on a very old distribution like CentOS 6 (2011!)

Comments, issues and PRs welcome on: https://github.com/metwork-framework/log_proxy

Fabien
  • 1