Creating an 'event log' from criteria across multiple column of data

KNIME Analytics Platform
1

Hello,
I have a data set that has several columns that I need to aggregate according to if one of the columns goes from ‘0’ to ‘1’. The ‘1’ flag denotes that an event is occurring, and the event can last a variable amount of time (minutes/hours). The data itself is recorded every second over several months…
Further, I need to flag if column C goes above 9.0 during the course of an event

Example data set.xlsx (11.5 KB)

What I have tried so far:

I have tried to use a Rule based Row filter ($EventCol$ >0 => TRUE) by if the Event column is 1/0, and then putting the data through a Date&TimeDifference node with a seconds granularity, calculating difference to the previous row.
Spot checking the data at this point doesn’t appear generate the correct results for an event, but I’m also not sure the best way to proceed to capture the Start Time, End Time, and Duration of an event…

If I were using another language, this would be a matter of running a For loop with some If/Elif statements and logging cell contents at the Start/End and during the event. I’m trying to move away from thinking in terms of individual cells, more in terms of across whole columns…

Note:
The data that I need would be organized into a new table with Event_#, Start time, End time, Duration of Event, Sum of A during event, Sum of B during event, Max value of C…

For some of these, I’m sure I can use the GroupBy node, but I’m new to KNIME, and unfamiliar with all the capability.

Thank you for reading.
Best,
Sp.Ag

2

Hello @SpectralAg,

and welcome to KNIME Community!

You need to detect when event happens and mark it with unique identifier. Then simply run GroupBy node to get wanted aggregations together with the Start Time (min) and End Time (max). To get duration use already mentioned Date&Time Difference node but calculate difference like Min column in relation to Max column.

To detect when change happens (event starts) I use approach from following workflow:

First lag, then add your logic into Rule Engine and finish it up with Missing Value node.

Br,
Ivan

1 Like
3

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.