What are some best practices when deploying KNIME within a Service Oriented Architecture (SOA)?
We've developed a collection of KNIME nodes and want to start deploying them as a service within our partner's environment. Our technology can be delivered in three ways:
(1) First, our large "Tuning Workflow" runs in the background and loads a set of well-defined results into a centralized repository. This Tuning Workflow takes about 20 hours to execute and can be scheduled to run either daily or weekly. A large client may need a cluster of 20 machines running this workflow non-stop.
(2) Second, our smaller "What If Workflow" can be run on demand when requested by a user. This What If Workflow leverages the earlier results from the Tuning Workflow and only needs 1 or 2 minutes to run. The results would be displayed in our partner's web application.
(3) Third, each of our individual nodes can be wrapped in tiny "Node Workflows" and called via an API as an integrated part of our partner's application. Our partner does NOT use KNIME and don't want to be responsible for a KNIME integration. Their application obviously expects an almost immediate response from our Node Workflows. Hence the full KNIME stack cannot be re-loaded in the background whenever a single node needs to be called. Large chunks of data can also not be sent across the internet. I think a persistent KNIME stack that sits as close to the partner application as possible is needed.
Deploying KNIME following a Service Oriented Architecture (SOA) would abstract our technology and insulate our partners and users from KNIME. They don't need to know that KNIME is doing a lot of the heavy lifting - nor do they care.
Any thoughts on best practices we should be following? I think I need the professional version of KNIME Server but I've not used it before and I am out of my depth. I asked the folks at KNIME this question but they haven't been able to get back to me.