I would like to know hot to edit the preferences for Spark in Knime-Server. I could run spark jobs successfully from the Analytics Platform because I could set the Spark-Job-Server URL through preferences > Kinme > Kinme Big Data Extensions > Spark.
But when running the workflow through the WebPortal I got the following error, because it tries to connect to the default URL (Localhost)
Execute failed: Spark Jobserver gave unexpected response (for details see View > Open KNIME log). Possible reason: Incompatible jobserver version, malconfigured Spark jobserver
How to set the correct URL in Knime Server ?
there are two ways to do this.
The recommended one to use the "Create Spark Context" node in your Spark workflows. This allows you to make the workflow more self-contained and independent of the default Spark context from the preference page.
The second way is to
- export your KNIME Preferences in the client (File > Export Preferences)
- open the resulting .epf file in a text editor
- locate the lines for the Spark jobserver
- copy/paste them to the preferences.epf on KNIME Server
- Restart KNIME Server.
Note however that now all workflows on KNIME Server will use that Spark context, unless these workflows use the Create Spark Context node.
Thanks alot ! It worked for me.