Automated correlation of log file data

Dear Team,

I'm working for a large Enterprise customer on solving a correlation problem across multiple system log files, typically in .csv however the columns and rows are all different e.g.

System A generates a request a1 ( log 1) > System B generates Order ID b1  (log 2) which contains the reuqest ID a1 > System C then generates order c1 where Order ID b1 is contained in sub orderline ID c1.1 (log3).....

There are many systems in the chain and also a one to many relationship in Order ID no.

We want to reverse engineer the concept of traceable hash e.g. a unique ID we can insert into all the collated log data to show allow us to trace the end to end process across the systems.

I am struggling to build a reader for each input log file ( parsing rows etc.) even before attemtpting to create some sort of correlation ID.

Is there a AI type module that could do this ?







Hi Werner,

I recommend to use the Web Log Reader node to parse log files. You an specify the log format in the node dialog and even specify time and date formatting.

Cheers, Kilian

Hi Werner,

On top of what Kilian said, I also wanted to point you to our training courses:

This could be a great way for you to improve your KNIME skills :-)