Cast exception after connecting 'create spark context' node to 'spark java snippet(source)' node

Hi,

I'm trying to read a file from a remote server. If I use 'spark java snippet(source)' node alone, and put server settings into File- >Preferences- >Spark, everything works fine. I can read the file into RDD.

Then, I put the server settings into a 'create spark context' node, the node can still connect to spark job server just fine. However, after I connected the 'create spark context' node with the 'spark java snippet(source)' node and executed the flow, I got a cast exception. What did I do wrong? Thanks in advance.

The log:

java.lang.ClassCastException: com.knime.bigdata.spark.port.context.SparkContextPortObject cannot be cast to com.knime.bigdata.spark.port.data.SparkDataPortObject
	at com.knime.bigdata.spark.node.scripting.java.AbstractSparkJavaSnippetNodeModel.executeInternal(AbstractSparkJavaSnippetNodeModel.java:164)
	at com.knime.bigdata.spark.node.scripting.java.source.SparkJavaSnippetSourceNodeModel.executeInternal(SparkJavaSnippetSourceNodeModel.java:82)
	at com.knime.bigdata.spark.node.SparkNodeModel.execute(SparkNodeModel.java:247)
	at org.knime.core.node.NodeModel.executeModel(NodeModel.java:563)
	at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1146)
	at org.knime.core.node.Node.execute(Node.java:933)
	at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:556)
	at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)
	at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:179)
	at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:110)
	at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:328)
	at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:204)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
	at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)

Hi,

you didn't do anything wrong. This is a bug in the Spark Java Snippet (Source) node. We will fix this problem with the first bug fix release. Unfortunately the release tomorrow will still have this problem since it is to late to fix it. Since the Spark Context input is optional you can use the node without an input which results in the node using the default Spark context defined in the Spark preferences. I hope this workaround is ok with you until the bug fix release.

Bye

Tobias

I see, thanks for the reply.