Automating Workflow Generation: Creating .knwf Files with File Readers Programmatically

KNIME Analytics Platform
1

Hello everyone!

I’m working on a tool that automatically generates a KNIME workflow (*.knwf) from a folder path by creating one File Reader (or CSV Reader) node per file in that folder. I’ve come up with two possible approaches:

  1. Manually Generating the Workflow Files

    • Essentially, I’d replicate how KNIME structures a workflow on disk.
    • I’d create a folder containing workflow.knime for the main workflow settings, plus subfolders under nodes/ (e.g., node_0/, node_1/, etc.) for each reader node’s settings.xml file.
    • After populating the settings for each node (pointing to the correct file path, delimiter info, etc.), I’d zip it all up and rename the zip to *.knwf.
    • This requires carefully mirroring KNIME’s internal file and folder layout but can be done without a running KNIME environment.

  2. Using KNIME’s Java API

    • I’d write code (e.g., in an Eclipse plugin or a headless KNIME application) that instantiates a new WorkflowManager, programmatically adds nodes (e.g. CSV Reader node containers), configures them (file path, delimiter, etc.), then uses the KNIME core classes (e.g. WorkflowPersistor, WorkflowSaveHelper) to write the workflow to disk.
    • This approach is “cleaner” as it stays within KNIME’s official APIs. However, it might require some knowledge of internal classes and typically needs to run inside a KNIME/Eclipse/OSGi environment.

I’d appreciate any thoughts or experiences with either method. Which do you think is more robust in the long term? Are there any pitfalls with direct file creation vs. using the KNIME API?

Thanks in advance for your help!

1 Like
2

You seem to be full of ideas… Can you maybe help me understand where you feel this may come in beneficial?

Also without being disrespectful - allow me to ask this question - please pick from the three option a - c below:

a) you are a Gen-AI powered bot
b) you are a Gen-AI assisted human
c) none of the above - are you kidding? What makes you think any of that?

Please answer honestly with a, b, or c.

1 Like
3

Hello,

Thank you for your thoughtful question! I’d like to be completely transparent with you—I use AI to help write my responses. My English level doesn’t always allow me to provide precise, technical answers with the necessary vocabulary, so the AI assists me in bridging that gap and ensuring clarity in my communication.

To address the broader context, I’m exploring this type of solution because it could automate a part of my work in the company where I’m employed. We manage numerous projects simultaneously and rely on diverse data sources. For each project, we often need to create a mini-ETL tailored to specific requirements, which can be repetitive and time-consuming. My task is to explore pre-developed approaches and adapt them to meet the technical criteria I’ve been given.

I hope this helps explain my interest in this topic, and I truly appreciate your engagement in the discussion!

Best regards,
Mathis

1 Like
4

Hi Mathis,

first of all apologies that I did that test on you - I’m part of the moderators and we observed some weird activity that appears to be entirely bot-driven and as the format and style of your posts looked a bit odd, I wanted to make sure.

I see where you are coming from with your issue - I’m thinking maybe you could solve your issue by using a component that reads all files in a folder?

That way it should be quick to point a component to a folder and then put the files through whatever process is defined.

It feels a bit complex to me to write code for a no code / low code tool… not sure if that makes sense.

I’m afraid I can’t really contribute to helping you solve this the way you intend :-).

1 Like
5

Hello Martin,

Thank you for your response and advice. I actually found a way to automatically generate a KNIME workflow using the source code. However, after some consideration, I decided it would be more efficient to use my time to create an effective workflow manually with the graphical interface of KNIME.

Thanks again for your help and suggestions!

Best regards,
Mathis

2 Likes