Knime Spark 1.5 CDH 5.5

KNIME Extensions
,
1

Hi,

Is Knime Spark Executor compatible with Spark 1.5 (CDH 5.5.x)?

Cheers,

Simon

2

Hi Simon,

as of today the KNIME Spark Executor is compatible with Spark 1.2 and 1.3. For more details about the supported versions have a look at the Compatibility section of the KNIME Spark Executor product page.

We will add support for Spark 1.4 and 1.5. with the next version of the executor which is planned to be released in Q1 2016.

Bye,

Tobias

3

I understand that KNIME is currently compatible with Spark 1.2, 1.3 only.

I was trying to use it with Spark 1.5 on CLoudera CDH5.5 and I got the following error whan I tried to execute Spark k-Means node. is that because of compatibility issue ? or there is another reason that might cause the error ?

[ERROR Spark k-Means 3:92 Execute failed: Job failure: "OK"]

If yes, then when the compatible version of KNIME Spark Executor will be realeased ?

4

Hello,

The error is caused by compatibility issues. Even though only the minor version of Spark has changed the Spark API has changed in a way that it is not backward compatible. That is why we are working on the next release of the KNIME Spark Executor which is planned for May. This release will add support for Spark 1.4 and 1.5.

Bye

Tobias

5

Hi,

I am also trying to use Spark nodes in KNIME. Whether Spark nodes can support only Cloudera distribution or it can access Apache Spark versions too.

Regards,

Prasanna

6

Hi,

for Apache Spark you can use the Hortonworks specific Spark Job Server e.g. the Spark Job Server for HDP 2.3 for Spark 1.3 since Hortonworks uses the standard Spark libraries. Cloudera on the other hand uses a Spark version with other library versions than the Apache Spark which is why we provide the two different Job Servers.

Bye

Tobias

7

Hi,

Thanks for your information. It is necessary that Spark Job Server must be installed in the machine we access and it is possible to install Spark Job Server in CDH 5.5 with Spark 1.5.0 version?

Regards,

Prasanna

8

Hi,

I got the same compatibility problem with the current version of Spark executor. We currently use Hortonwork HDP 2.4 with Spark 1.6.0. May I know if the new version of Spark executor will support the Spark 1.6 as well?

Thanks,

9

Hi,

support for Spark 1.6 is also planned for the next version of the KNIME Spark Executor.

Bye

Tobias

10

Hi Tobias,

We're really looking forward to the next release for the next KNIME spark executor; any updates on when that is planned for? We're running a cdh 5.5 cluster with spark 1.5

Thanks,

11

Hi ouyangze,

the new KNIME Spark Executor is coming out with KNIME Analytics Platform 3.2, which is en-route to be released this Friday (July 8th 2016).

Best,

Björn

12

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.