SLF4J: Different Log Files for Different Log Levels

When we have a product with lots of output flying around, following logs can get really hard, especially under heavy traffic. To lower the struggle, we may want to have different log files for different log levels. In that way, reading through INFO logs would be enough for regular checks. And when we have some wrong/unwanted behavior we can check the DEBUG file. We can throw alerts for each line in ERROR file. It is all about the use-case. If you have a relatively small application with not so much logs, splitting them may not be as effective.

I came across with such situation while developing a Spring Boot application for a client. The application is expected to go through heavy traffic and, their operation team wants to separate transaction logs from other logs. So I searched through Slf4j and Logback documentation and implemented the desired behavior. Creating a Logback file with related attributes is enough to reach above goal without changing any code.

How to do it?

For some configurations like logging pattern and logging level, modifying the application.properties file is enough. But to reach the above goal, we need to configure Logback. If we put a logback.xml in the root of our classpath, it is picked up from there (or from logback-spring.xml, to take advantage of the templating features provided by Boot). Spring Boot provides a default base configuration that we can include.

We will use Appenders to set different file appender for each logging level.

What is an Appender?

Logback delegates the task of writing a logging event to components called appenders. Appenders are ultimately responsible for outputting logging events. However, they may delegate the actual formatting of the event to a Layout or to an Encoder object. I will not go into many details as this is not a Logback tutorial. To learn more about LOGBack, check the Logback Manual.

Example Logback file:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>

    <include resource="org/springframework/boot/logging/logback/defaults.xml"/>

    <property name="LOG_FILE_INFO" value="${LOG_FILE_INFO:-${LOG_PATH:-${LOG_TEMP:-${java.io.tmpdir:-/tmp}}/}/info.log}"/>
    <property name="LOG_FILE_ERROR" value="${LOG_FILE_ERROR:-${LOG_PATH:-${LOG_TEMP:-${java.io.tmpdir:-/tmp}}/}/error.log}"/>

    <include resource="org/springframework/boot/logging/logback/console-appender.xml"/>

    <appender name="LOG_INFO" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${LOG_FILE_INFO}</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${LOG_FILE_INFO}.%d{yyyy-MM-dd}.log</fileNamePattern>
        <maxHistory>30</maxHistory>
        </rollingPolicy>
    <filter class="ch.qos.logback.classic.filter.LevelFilter">
        <level>INFO</level>
        <onMatch>ACCEPT</onMatch>
        <onMismatch>DENY</onMismatch>
    </filter>
        <encoder>
            <pattern>%d{dd-MM-yyyy HH:mm:ss.SSSS}|%-5level|%msg%n</pattern>
        </encoder>
    </appender>

    <appender name="LOG_ERROR" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${LOG_FILE_ERROR}</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${LOG_FILE_ERROR}.%d{yyyy-MM-dd}.log</fileNamePattern>
        <maxHistory>30</maxHistory>
        </rollingPolicy>
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
        <level>WARN</level>
    </filter>
        <encoder>
            <pattern>%d{dd-MM-yyyy HH:mm:ss.SSSS}|%-5level|%msg%n</pattern>
        </encoder>
    </appender>

    <root level="INFO">
       <appender-ref ref="CONSOLE"/>
    </root>

    <logger name="com.mypackage.myapplication" additivity="false" level="ALL">
       <appender-ref ref="CONSOLE"/>
       <appender-ref ref="LOG_INFO"/>
       <appender-ref ref="LOG_ERROR"/>
    </logger>

</configuration>

In the above example, I created two different Rolling File Appender. They are both using TimeBasedRollingPolicy which is probably most popular rolling policy. For other rolling policies and detailed explanations check here. Also, notice that each appender has a different File attribute which is required use different files for different appenders.

The key part for writing different log files for different log levels is using filters. To make the Appender accept only a specific logging level, we should use a level filter that will accept the matched logs and deny the others. To make the Appender accept every log above a specific level, we can use ThresholdFilter. For example in the above example, Threshold filter with the level set to WARN will accept logs from WARN, ERROR and FATAL levels. Check the below image to have a better idea about logging level hierarchy.

Custom Loggers

After configuring the Appenders, we need to reference them in a logger. Normally referencing the appender in the root logger is enough, but if we want to change the log level for a particular package or class and specify other properties that are unique to that class we can create a custom logger. In the above example, I have used only console appender for the root logger. And I have used the custom appenders in my custom application specific logger.

This is it for this tutorial. The same method is valid for creating different log files for different log levels both in SLF4J and LOG4J with a little modification in logback.xml. Check their documentation for a better overview of what you can or can not do.
Thank you for reading so far, see you next time!

If you liked the content please share it!

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.