How Does KNIME Handle Real-Time Data When Using Kafka Nodes?

KNIME Analytics Platform
,
1

Hello KNIME Team and Community,

I have a question regarding the use of Kafka-related nodes in KNIME, from both a user and an operator perspective.

As far as I understand, KNIME provides nodes that can connect to Kafka. Since Kafka continuously accumulates data in real time, I would like to clarify how this behavior is handled when using Kafka nodes in KNIME.

  1. When a Kafka node is executed in KNIME, does it retrieve only the data that has been accumulated up to the moment the node execution succeeds?

  2. From a customer or end-user perspective, if they want to view data that is being generated every 1 second or every 10 seconds in near real time, is the best (or only) approach in KNIME to schedule workflows as batch jobs that run every 1 or 10 seconds to fetch the data?

I would appreciate your guidance on these two questions.

Thank you in advance for your support.

2

Does anyone happen to know the answer to this question?

I would really appreciate any response.