Please tell me how to set up distributed execution of Knime tasks on Apache Spark. I would like to implement automatic task management and distribution of tasks across the nodes of the home LAN. I would like to do without the knime server and other paid servers. As I understand it, the required can be implemented via Apache Spark Standalone. But how? My node is two ordinary computers. I deployed Knime and Spark on both Windows computers, I configured master and slave. The Spark web interface perfectly sees the worker on the slave. I’m running a workflow with Spark nodes. They work. But there is no distribution of tasks to the slave. What do I need to do to make this happen?
Thank you for your reply. I have Knime running under Windows, all the instructions for your link and a lot of links to the Internet are made for Linux. The second point concerning Livy, including, at least what I see from the links, all this requires working with products under a paid license. My project is still non-commercial and more for self-development and self-education, so it is still economically unjustified for me to use paid licenses.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.