I’m working on a workflow where I need to process a variable number of input tables using a Python Script node. Currently, I’m adding one input table at a time, but I’m wondering if there’s a more elegant solution.
Is there a more elegant method to handle a variable number of input channels in a Python Script node than adding one more input table at a time? If so, how are such input channels accessed in Python?
Current Approach
Right now, I’m manually adding input ports to the Python Script node for each new table I need to process. This works, but it feels cumbersome and not very flexible.
What I’m Looking For
A method to dynamically handle any number of input tables without changing the node configuration.
The Python code to access these variable input channels within the script if the access differs from the usual methods.
Any insights, best practices, or code examples would be greatly appreciated. I’m particularly interested in solutions that scale well and require minimal changes when the number of input tables changes.
@ekadagami question is if the files all have the same structure. If yes you could put them in a folder as parquet files and address them as one file in Python.
In general parquet or SQLite could be formats to communicate between KNIME and Python. SQLite could store several tables and the tables can be accessed within Python / KNIME nodes. Question is if the code to handle them would be based on the structure of the table.
Thank you for your answers. I realize that I may not have worded my question correctly. Instead of input tables, I should have said nodes. So I want to connect n nodes to the input of my Python Script Node without having to change the configuration of my Python Node for each new connection (adding an input node via ⨁)
