I am still so confused by Knime Server vs the New Model and need some help figuring out what I need.
I will have one Machine were I plan to build components and workflows on (PC). Test them, debug them etc. Then I would like to push these workflows on a few different (4 machines) that are Compute Instances on Google Cloud, kind of like “workers”. 90% of these workflows are scheduled on should be running daily, hourly etc.
Today I have them all running on a local machine using wait nodes… But when I have to restart or the machine crashes I forget to relaunch them etc. All of the Workflows either collect data via APIs, parse it and save it in a Google Cloud DB instance or Scrape, download content, archive and write into DB.
How would I do that best? I am thinking
- Local Machine Creates components and workflows and then uploads them HUB.
- One “scheduler” that has a workflow that uses timers to execute remote workflows
- Multiple workers that (unclear on this) have workflows that call workflows from myHub, run them and then ?
Again the objective here is to be able to have a version control of sorts as I update components and multiple workers executing them against a schedule.
Ideas? What am I missing?