We are starting to think in buy the KNIME Server, but we need to be really sure that have advantages like be able to run for example 5 workflows at same time in parallel.
because we have this kind of problems at the moment, we Have a workflow job running every 15 minutes and sometimes this provoke some other failed workflows/jobs. In future we will have many problems like this.
At the moment we have around +20 jobs/workflows but maybe in 6 months will have +100 jobs.
Any recommendation to setup a robust environment for this?
We are using AWS and Windows server 2016 in EC2 and Rundeck to manage Jobs scheduling.
And what about running Knime as Cluster? Seems like this only work in Linux.
This will be a smooth migration without problems with the +30 workflows we already have?
Thinking to Buy the KNIME Server if this is really the best option but don’t want to go for Linux.
Hi Peter,
My colleagues should have been in touch about the commercial details for KNIME Server.
The KNIME Server offering on the AWS Marketplace is currently for KNIME Server Small, or KNIME Server Medium. They are preconfigured AMIs which you correctly noted are running Linux.
We don’t currently have plans to offer preconfigured AMIs through the AWS Marketplace that are based on Windows Server. However, KNIME Server is available to install and manage yourself on any supported OS. Windows Server 2016 is a supported OS, and there is an installation guide available here which gives some idea of the complexity.
https://docs.knime.com/latest/server_installation_guide/index.html
In the case that your current ~20 jobs can be run on a single large AWS instance, then your initial setup for KNIME Server should allow you to run those jobs straightaway with the simplest setup. In the case that an individual instance becomes a limiting factor for scaling of workflow execution, then you may choose to use the KNIME Server distributed executors. In that case additional executors installed on a separate instance can communicate with the KNIME Server to provide more workflow compute power. In our experience, managing that kind of setup is easier to do using Linux, and we plan to release a KNIME Server Large (with distributed executors) offering on the AWS Marketplace fairly shortly. However, it is also possible to use Windows Server to achieve the same setup.
Hope that helps,
Jon
1 Like