48

I am having trouble with duplicate log messages when writing to multiple log files using log4j.

At present I am trying to log INFO level data (and upwards) for the specific logger named foobar in my foo.log file and then all WARN level log messages (and upwards) for all loggers in the bar.log file.

As a result of this, duplicate log messages were written to the foo.log file (each line was logged twice) and after some quick research I found that the suggestion to fix this was to add log4j.additivity.foobar=false to my properties file.

The problem with this is that although it stops duplicate lines, the WARN message from the foobar logger are never written to the bar.log file.

My log4j properties file is as follows:

log4j.rootLogger = WARN, FOO, BAR
log4j.logger.foobar = INFO, FOO
log4j.additivity.foobar = false

log4j.appender.FOO = org.apache.log4j.RollingFileAppender
log4j.appender.FOO.layout = org.apache.log4j.PatternLayout
log4j.appender.FOO.layout.ConversionPattern = %d{ISO8601} %-5p %c ~ %m%n
log4j.appender.FOO.File = foo.log

log4j.appender.BAR = org.apache.log4j.RollingFileAppender
log4j.appender.BAR.layout = org.apache.log4j.PatternLayout
log4j.appender.BAR.layout.ConversionPattern = %d{ISO8601} %-5p %c ~ %m%n
log4j.appender.BAR.File = bar.log

Does anyone know how I can write the log messages to both log files (as it was doing before I started setting the additivity property) and still prevent the duplicate log messages?

Please note, this is a simplified summary of the problem. In the real world scenario there are multiple loggers and more than two log files

My Head Hurts
  • 37,315
  • 16
  • 75
  • 117

1 Answers1

63

This problem can be solved in two parts.

1. Prevent duplicate log messages

The log messages were written twice because we listed the FOO appender in both the rootLogger and the log4j.logger.foobar category. So we must remove the appender and only define the logging level in category:

log4j.rootLogger = WARN, FOO, BAR
log4j.logger.foobar = INFO

This means that the INFO level messages from log4j.logger.foobar are propagated upwards to ALL of the loggers the appenders in rootLogger, but will only be written to each log file once.

2. Prevent INFO level message being written to bar.log

Since all of the INFO level log messages for the log4j.logger.foobar category are being inherited by the appenders in rootLogger, we need to stop the BAR appender for recording the INFO level messages.

We can achieve this by setting the Threshold property on the BAR appender itself:

log4j.appender.BAR.Threshold = WARN

This will prevent the INFO level statements being logged in the bar.log file as it will only accept levels of WARN and upwards.

So the complete log4j properties file would be as follows:

log4j.rootLogger = WARN, FOO, BAR
log4j.logger.foobar = INFO

log4j.appender.FOO = org.apache.log4j.RollingFileAppender
log4j.appender.FOO.layout = org.apache.log4j.PatternLayout
log4j.appender.FOO.layout.ConversionPattern = %d{ISO8601} %-5p %c ~ %m%n
log4j.appender.FOO.File = foo.log
log4j.appender.FOO.Threshold = INFO

log4j.appender.BAR = org.apache.log4j.RollingFileAppender
log4j.appender.BAR.layout = org.apache.log4j.PatternLayout
log4j.appender.BAR.layout.ConversionPattern = %d{ISO8601} %-5p %c ~ %m%n
log4j.appender.BAR.File = bar.log
log4j.appender.BAR.Threshold = WARN
My Head Hurts
  • 37,315
  • 16
  • 75
  • 117
  • this is all nice and well, but this is not what was asked. How does this differentiate by loggers? the original question said only `foobar` goes to foo, and all loggers including foobar goes to bar – f.khantsis Jan 19 '17 at 02:31
  • The question asks for a solution where all `INFO` messages for the `foobar` logger goes to `foo.log` file and all `WARN` messages for all loggers goes to `bar.log` file (and also that each log message is only written once in the appropriate file). – My Head Hurts Jan 20 '17 at 14:47
  • no, __all loggers__, not just `foobar` get sent to `bar.log`. As per the question: "At present I am trying to log `INFO` level data for the specific logger named `foobar` in my foo.log file and then all `WARN` level log messages for __all loggers__ in the bar.log file." – f.khantsis Jan 22 '17 at 00:20
  • I don't see what we are saying differently here - except in your first sentence you don't acknowledge that **only** `WARN` level messages for **all loggers** get sent to `bar.log` (but you do in the copy & paste extract from my original question). – My Head Hurts Jan 23 '17 at 07:53
  • yea, in the solution __all loggers__ go to `foo.log`, which the question only asked that only __foobar__ is logged to `foo.log`. Basically, if there is a `WARN` message for `barfoo` logger, it should ___NOT___ go to __foobar__ – f.khantsis Jan 23 '17 at 17:59
  • OK, I understand now what you mean. When asking my question I wanted to log `INFO` level and upwards within my **foobar** logger and only `WARN` level and upwards in my other loggers. I will amend my question so that is clear. – My Head Hurts Jan 24 '17 at 09:23
  • Even so it's not clear, does a `WARN` level message for __barfoo__ logger go to `foo.log`? – f.khantsis Jan 24 '17 at 09:28
  • I haven't worked with Java or log4j for about 3 years and it has been over 4 years since I asked and answered this question. Looking at the answer and question, I would say that `WARN` level messages for a hypothetical logger called **barfoo** would be intended to go to `bar.log`, but not `foo.log`. But this of course would depend on how you set it up. – My Head Hurts Jan 24 '17 at 09:37
  • well, according to your answer, __barfoo__ would go to both `FOO` and `BAR` appenders, meaning it would go to both `bar.log` and `foo.log` – f.khantsis Jan 24 '17 at 17:56
  • as a more real world example, I want to have two logfiles, one for general logging, and one for a specific component I am working with . The specific component logfile must ONLY have log entrys for that component, and must have detailed logs, while the general logfile has log entries for all components, including the specific one, and should only show problems – f.khantsis Jan 24 '17 at 18:22
  • I think you might be better served if you open a new question on Stackoverflow. I imagine there are people who will have more recent knowledge / experience who can guide you more than I can in these comments :) – My Head Hurts Jan 24 '17 at 21:40