Switch log level from warn to info for empty table messages

Hi,

in order to reduce log noise and to improve debugging comfort I want to suggest to switch the log level of empty tables being returned from warn to info.

In most of the cases an empty table isn’t a problem but actually intended or the default result.

Time Stamp Message
2024-09-19 12:07:10,497 WARN Java Edit Variable 3:1141:0:1122:0:1120 The flow variable file.separator is not found in the input.
2024-09-19 12:07:10,516 WARN Java Edit Variable 3:1170:1141:0:1122:0:1120 The flow variable file.separator is not found in the input.
2024-09-19 12:07:10,563 WARN List Files/Folders 3:1141:0:1124 Node created an empty data table.
2024-09-19 12:07:10,563 WARN List Files/Folders 3:1170:1141:0:1124 Node created an empty data table.
2024-09-19 12:07:10,564 WARN Empty Table Switch 3:1141:0:1131 Node created an empty data table.
2024-09-19 12:07:10,564 WARN Empty Table Switch 3:1170:1141:0:1131 Node created an empty data table.
2024-09-19 12:07:10,564 WARN Column Filter 3:1141:0:1132 Node created an empty data table.
2024-09-19 12:07:10,565 WARN Column Filter 3:1170:1141:0:1132 Node created an empty data table.
2024-09-19 12:07:10,567 WARN NoOp (Table) 3:1141:0:1111 Node created an empty data table.
2024-09-19 12:07:10,568 WARN NoOp (Table) 3:1170:1141:0:1111 Node created an empty data table.
2024-09-19 12:07:10,568 WARN End IF 3:1141:0:1110 Node created an empty data table.
2024-09-19 12:07:10,568 WARN End IF 3:1170:1141:0:1110 Node created an empty data table.

Though, it might be worth to think about adding an option, which can be enable on demand in node config, to throw an actual warning i.e. if a row (reference) or rule based row filter gave an empty table back.

PS: As follows my log4f adjustment too filter out these messages:

File location (for me): E:\Knime-Workspace.metadata\knime\log4j3.xml

<!-- Filter "Node created an empty data table." -->
    <appender name="stdout" class="org.apache.log4j.ConsoleAppender">
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%-5p\t %t %c{1}\t %.10000m\n" />
        </layout>
        <filter class="org.apache.log4j.varia.StringMatchFilter">
            <param name="StringToMatch" value="Node created an empty data table." />
            <param name="AcceptOnMatch" value="false" />
        </filter>
        <filter class="org.apache.log4j.varia.DenyAllFilter"/>
    </appender>

PS: The fallback seems to have improved but not fully filtered all out :thinking:

PPS: Nope, the filter didn’t filter anything :frowning:

Best
Mike

Are you trying to filter these messages from

  1. the Classic KNIME Console View,
  2. from the logfile, or
  3. from the node message/ModernUI Workflow Monitor?

Because you configure the appender name="stdout" it targets the output that you can see when you start the binary from a terminal (or from Eclipse SDK). It might be confusing, because the class is called ConsoleAppender.

The one you robably want to mess around with is appender name="knimeConsole", so 1. in the list above.
2. in the list above would be appender name="logfile".

The node message/Modern UI Workflow Monitor cannot be filtered.

2 Likes

I want to filter the Classic KNIME Console View.

Then you need to target appender name="knimeConsole" in the file. Does it work?

Catching up to speed. Nope, unfortunately it did not resolve it. Maybe I missed something since almost two months have passed (sorry for the delayed response)

Here is my full log4j3.xml

<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
    <appender name="stdout" class="org.apache.log4j.ConsoleAppender">
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%-5p\t %t %c{1}\t %.10000m\n" />
        </layout>
        <filter class="org.apache.log4j.varia.LevelRangeFilter">
            <param name="levelMin" value="WARN" />
            <param name="levelMax" value="WARN" />
        </filter>
    </appender>

    <appender name="stderr" class="org.apache.log4j.ConsoleAppender">
        <param name="Target" value="System.err" />
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%-5p\t %t %c{1}\t %.10000m\n" />
        </layout>
        <filter class="org.apache.log4j.varia.LevelRangeFilter">
            <param name="levelMin" value="ERROR" />
        </filter>
    </appender>

    <appender name="batchexec" class="org.apache.log4j.ConsoleAppender">
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%-5p\t %t %c{1}\t %.10000m\n" />
        </layout>
        <filter class="org.apache.log4j.varia.LevelRangeFilter">
            <param name="levelMin" value="INFO" />
            <param name="levelMax" value="INFO" />
        </filter>
    </appender>

    <appender name="logfile" class="org.knime.core.util.LogfileAppender">
        <layout class="org.knime.core.node.NodeLoggerPatternLayout">
            <!--  Supported parameter are 
                %W = Workflow directory if available
                    the depth can be specified e.g. %W{2} prints the name of the workflow and the folder it is located in
                    if the depth is omitted the complete workflow directory is printed
                %N = Node name if available
                %Q = Qualifier as a combination of N (Node name) and c (category). Displays the category if the node name is not available.
                    the depth can be specified and is used for the category e.g. %Q{1} prints only last index of the category
                    if the depth is omitted the complete category is printed
                %I = Node id if available
                    the depth can be specified e.g. %I{1} prints only last index of the node id
                    if the depth is omitted the complete node id is printed
                %J = Job id if available
            -->
            <param name="ConversionPattern" value="%d{ISO8601} : %-5p : %t : %J : %c{1} : %N : %I : %m%n" />
        </layout>
    </appender>
    
    <appender name="knimeConsole" class="org.apache.log4j.varia.NullAppender">
        <!-- This appender is only used to retrieve the conversion pattern layout for the KNIME console output
        that is why we use the NullApender that does no log to any file. -->
        <layout class="org.knime.core.node.NodeLoggerPatternLayout">
            <!--  Supported parameter are 
                %W = Workflow directory if available
                    the depth can be specified e.g. %W{2} prints the name of the workflow and the folder it is located in
                    if the depth is omitted the complete workflow directory is printed
                %N = Node name if available
                %Q = Qualifier as a combination of N (Node name) and c (category). Displays the category if the node name is not available.
                    the depth can be specified and is used for the category e.g. %Q{1} prints only last index of the category
                    if the depth is omitted the complete category is printed
                %I = Node id if available
                    the depth can be specified e.g. %I{1} prints only last index of the node id
                    if the depth is omitted the complete node id is printed
                %J = Job id if available
            -->
            <param name="ConversionPattern" value="%d{ISO8601}\t%-5p %-20Q{1} %-10I %.10000m\n" />
            <!-- Strict pattern that maintains the distance by truncating long qualifiers and node ids
            <param name="ConversionPattern" value="%-5p %-35.35Q{1} %-20.20I %.10000m\n" />
            -->
            <!-- Compact log pattern.
            <param name="ConversionPattern" value="%-5p %Q{1} %I %.10000m\n" />
            -->
        </layout>
    </appender>

<!-- Filter "Node created an empty data table." -->
    <appender name="knimeConsole" class="org.apache.log4j.ConsoleAppender">
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%-5p\t %t %c{1}\t %.10000m\n" />
        </layout>
        <filter class="org.apache.log4j.varia.StringMatchFilter">
            <param name="StringToMatch" value="Node created an empty data table." />
            <param name="AcceptOnMatch" value="false" />
        </filter>
        <filter class="org.apache.log4j.varia.DenyAllFilter"/>
    </appender>


<!--
if you want to enable debug message for a specific package or class, add something like:

<logger name="org.knime.dev.node.xyz">
    <appender-ref ref="debug"/>
</logger>
-->

    <logger name="org.knime">
        <level value="all" />
    </logger>

    <logger name="com.knime">
        <level value="all" />
    </logger>

    <logger name="org.knime.core.node.workflow.BatchExecutor">
        <appender-ref ref="batchexec" />
    </logger>

    <logger name="com.knime.product.headless.ReportBatchExecutor">
        <appender-ref ref="batchexec" />
    </logger>

    <root>
        <level value="error" />
        <appender-ref ref="stdout" />
        <appender-ref ref="stderr" />
        <appender-ref ref="logfile" />
        <appender-ref ref="knimeConsole" />
    </root>
</log4j:configuration>```

Ah, you’re defining consoleAppender twice:

  1. with <appender name="knimeConsole" class="org.apache.log4j.varia.NullAppender">
  2. with <appender name="knimeConsole" class="org.apache.log4j.ConsoleAppender">

I think you should be able to comment out the first element completely and it should work. It looks like it uses the first occurrence of the appender by name.