We are currently in the process of migrating from a KNIME Server to a KNIME Business Hub, and one of the key features we want to use with the Hub is the Trigger Deployment (to use an event listener starting a workflow when a file arrives).
As this only works with spaces, I am looking for a way to automate files uploading to a space using python. To provide some context, we are currently using an Airflow DAG to retrieve files from mailboxes and to upload them to our storage (Azure Blob Storage), where they are read by our workflows when the jobs run.
The idea here would be to change that Airflow process so that instead of uploading new files to Azure, it will send them to a dedicated Space in our Business Hub, and then trigger the execution of deployed workflows.
Unfortunately I didn’t find a way to do this yet. I’m not sure there is a python package that can connect to a Business Hub, but maybe we could do it with an API Post request ?
Thanks for your insight, as a reminder, the point here is to NOT use KNIME to do this as it would add some very heavy overhead on the whole process ^^
You can use KNIME Hub’s REST API to upload files. See https://api./api-doc/ for the API documentation. The service you are looking for is the catalog-service.
Note that you first have to create an Application Password in order to make (authenticated) API calls.
Hi,
To expand on @thor’s answer, you will need the
PUT https://api.yoursever.com/repository/{path}:data
endpoint. The path consists of the team name (URL encoded) or ID + the space name + any folder within the space where you want to put the items + the actual item name.
An example would be:
PUT https://api.yoursever.com/repository/my%20team/my%20space/my%20folder/my%20file.csv:data
To authenticate the call via Application Password, go to your user’s profile on Hub, then on the left side to Settings → Application Passwords and generate a new one there. This you can then use as basic authentication (username and password) in your REST calls.
Kind regards,
Alexander
That was a bit more complicated than anticipated by it’s working now, the tricky part is that the folder, if it doesn’t exist yet, has to be created in a separated POST request.
