Dynamically Loading CSV Paths into KNIME Workflow from Python API

KNIME Analytics Platform
1

Hi KNIME Community,

I’m working on a project where KNIME plays a key role in data processing for a service connected to an API. The use case involves a button in our application that, when clicked, triggers a process to load CSV files from 16 different databases dynamically, process the data in various ways using a KNIME workflow, and finally send the results to Neo4j.

I have already created the KNIME workflow, and I’m using the Neo4j extension, which allows me to load CSV files via the KNIME graphical interface, process them, and output the results into Neo4j. This part works seamlessly.

My main question: How can I configure the workflow to accept these CSV paths as variables or inputs dynamically from the API? Ideally, I’d like to trigger the workflow execution programmatically from Python and pass the file paths as parameters.

Any guidance, examples, or best practices for achieving this would be greatly appreciated!

Thank you!

2

Hey there and welcome to the forum!

Looks like you have managed a fair bit of complexity already.

There are ways how you can execute Knime workflows w/o running the graphical interface.

The best resource I know of is an article from @mlauber71 on Medium:

This might be good for you to go through as it covers what I think you want to do.

1 Like
3

Hi Martin,

Thank you for the warm welcome and the recommendation! I’ll definitely check out the article from @mlauber71 — it sounds like a great resource.

From what I’ve understood so far, there seem to be three main approaches to executing KNIME workflows without the graphical interface:

  1. Using an API via KNIME Business Hub: I’ve seen mentions of this, but I haven’t explored it in detail yet.
  2. Using subprocesses: This works regardless of the programming language, and it’s the approach I’m currently using.
  3. Using the knimepy library: I’ve come across this as well, but I’m unsure if it’s still actively maintained or widely used.

If there are other approaches or nuances I’ve missed, feel free to share — I’m eager to learn!

Thanks again!

4

I think that is fairly comprehensive. I was not even aware of knimepy.

1 Like