The webinar Best Practices to Build KNIME Workflows is happening on Wednesday, February 16, 2022 at 5 PM - 6 PM UTC +1 (Berlin) which is 10 AM - 11 PM UTC -6 (Chicago).
You engineer an ETL data pipeline that should work regularly, automatically, error-free, and smoothly. You have now created your workflow and during testing it seems to work well. But is it really ready to be put into use?
Is its design efficient? Will it work with new data? Will it scale? What will happen in the case of failure? Will one be able to trace an error easily? Finally, is it secure?
These questions are important to raise even if your workflows are very simple. In this webinar, we will share the best practices for data engineers to build efficient, robust, and secure workflows, regardless of their complexity.
Thank you for joining our webinar yesterday. Here are the materials of the webinar:
We didn’t manage to answer some of your questions yesterday? Please post them here! We will be happy to answer them.
thank you very much for this very helpful webinar!
I had two questions:
The first was related to security: How can I give others access to a workflow using Microsoft Authentication via KNIME Server web interface? How can they give their credentials? It needs sign-in in an additional browser window, and the Microsoft Authentication node does not accept the credential node input as far as I know. So far, I’m providing the workflow with my credentials as without read permissions, but I would like to remove my credentials from the workflow.
The second questions is about one of the reasons I’ve not used the call workflow service nodes so far: How can I share workflows that use Call Workflow Service on the KNIME Hub? Upload all workflows individually, or is it possible to upload them in a bundle? How can I make sure the user has all necessary workflows available? Or should I stay with using components in this case?
Thank you for attending the webinar and for your questions.
Apart from interactive authentication, Microsoft Authentication node has several other options how to provide credentials, for example, an option called Username/password authentication. It supports the flow variable from the Credentials Configuration node and, what’s more relevant if you work with the Web Portal, Credentials Widget node. If you wrap the Credentials Widget node into a component, a user of the application on the Web Portal can provide their credentials that will be then passed to the Microsoft Authentication node.
It depends on where your callee workflows are located.
- If they are located on your local machine, other users won’t be able to access them and you will need to upload all of them to the KNIME Hub. In this case, you could locate all of your callee workflows in the same workflow group and upload the whole workflow group to the KNIME Hub.
- If they are located on the KNIME Server, you can use the KNIME Server Connector node together with the Call Workflow Service node, then the users with the access to this server will be able to browse and access the callee workflows from the KNIME Server.
Does this help? Let us know in the case of further questions.
@daniela_digles A note regarding Workflow Services vs. Shared Components.
- Calling another workflow triggers its execution. A callee workflow doesn’t necessarily contain some standard reusable small process but rather performs some particular task to allow for modularity and to unload your caller workflow. See more details on Workflow Services here.
- Executing an instance of a shared component executes all the nodes wrapped into the component inside of the workflow where you execute this instance and doesn’t execute your shared component. Shared components are more suitable for small, standard, reusable, and repeatable processes and have more functionality available, e.g., configuration dialogs and composite views. See more details about components here.
Check which solution fits your use case best.