Spark preferences in Knime Server

KNIME Extensions
,
1

Hi,

I would like to know hot to edit the preferences for Spark in Knime-Server. I could run spark jobs successfully from the Analytics Platform because I could set the Spark-Job-Server URL through preferences > Kinme > Kinme Big Data Extensions > Spark.

But when running the workflow through the WebPortal I got the following error, because it tries to connect to the default URL (Localhost) 

Execute failed: Spark Jobserver gave unexpected response (for details see View  > Open KNIME log). Possible reason: Incompatible jobserver version, malconfigured Spark jobserver

How to set the correct URL in Knime Server ?

Thanks,

AAL

2

Hi AAL,

there are two ways to do this.

The recommended one to use the "Create Spark Context" node in your Spark workflows. This allows you to make the workflow more self-contained and independent of the default Spark context from the preference page.

The second way is to

  1. export your KNIME Preferences in the client (File > Export Preferences)
  2. open the resulting .epf file in a text editor
  3. locate the lines for the Spark jobserver
  4. copy/paste them to the preferences.epf on KNIME Server
  5. Restart KNIME Server.

Note however that now all workflows on KNIME Server will use that Spark context, unless these workflows use the Create Spark Context node.

Best Regards,

Björn

3

Thanks alot ! It worked for me.

4

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.