# Spark PMML Model Predictor

**URL:** <https://forum.knime.com/t/spark-pmml-model-predictor/15191>\
**Category:** KNIME Extensions\
**Tags:** big-data-extension, big-data\
**Created:** [May 15, 2019, 3:40pm UTC](https://forum.knime.com/t/spark-pmml-model-predictor/15191 "2019-05-15T15:40:07Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Aaron\_Hart](https://forum-cdn.knime.com/user_avatar/forum.knime.com/aaron_hart/32/7814_2.png) [@Aaron\_Hart](https://forum.knime.com/u/Aaron_Hart)\
**Post date:** [May 15, 2019, 3:40pm UTC](https://forum.knime.com/t/spark-pmml-model-predictor/15191/1 "2019-05-15T15:40:07Z")

</div>

I am trying to apply a KNIME generated k-means model to a spark dataframe but am failing because the pmml compiler throws an exception as a result of the evaluate method being \> 64k in size. My Model is built on 28 columns and fails if the k parameter is ~\>50. I use a meta-clustering approach so in production I would be using a k of ~ 256. Is this fixable? Stack trace is listed below.

Best,

Aaron

> ```
> Unable to compile expression
> ERROR at line 89
> The code of method evaluate(MainModel.Model_0_1.Data) is exceeding the 65535 bytes limit
> Line : 88 }
> Line : 89 public static Object[] evaluate(Data data) {
> at com.knime.pmml.compilation.java.compile.PMMLCompilerNodeModel.execute(PMMLCompilerNodeModel.java:98)
> at org.knime.core.node.NodeModel.executeModel(NodeModel.java:567)
> at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1186)
> at org.knime.core.node.Node.execute(Node.java:973)
> at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:559)
> at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)
> at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:179)
> at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:110)
> at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:328)
> at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:204)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
> at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)
> Caused by: org.knime.ext.sun.nodes.script.compile.CompilationFailedException: Unable to compile expression
> ERROR at line 89
> The code of method evaluate(MainModel.Model_0_1.Data) is exceeding the 65535 bytes limit
> Line : 88 }
> Line : 89 public static Object[] evaluate(Data data) {
> at org.knime.ext.sun.nodes.script.compile.JavaCodeCompiler.compile(JavaCodeCompiler.java:330)
> at com.knime.pmml.compilation.java.compile.CompiledModelPortObject.compileModel(CompiledModelPortObject.java:174)
> at com.knime.pmml.compilation.java.compile.CompiledModelPortObject.setCode(CompiledModelPortObject.java:149)
> at com.knime.pmml.compilation.java.compile.CompiledModelPortObject.<init>(CompiledModelPortObject.java:94)
> at com.knime.pmml.compilation.java.compile.PMMLCompilerNodeModel.execute(PMMLCompilerNodeModel.java:95)
> ... 13 more
> 
> ```

---

<div class="post-metadata">

**Author:** ![mlauber71](https://forum-cdn.knime.com/user_avatar/forum.knime.com/mlauber71/32/73590_2.png) [@mlauber71](https://forum.knime.com/u/mlauber71)\
**Post date:** [May 17, 2019, 2:03pm UTC](https://forum.knime.com/t/spark-pmml-model-predictor/15191/2 "2019-05-17T14:03:52Z")

</div>

It is not the same but have you thought about using H2O.ai K-means and sparkling water.

> **[H2O k-Means — NodePit](https://nodepit.com/node/org.knime.ext.h2o.nodes.learner.cluster.kmeans.H2OKMeansNodeFactory2)**
>
> Build and apply a K-Means clustering model using H2O.

> **[Create H2O Sparkling Water Context — NodePit](https://nodepit.com/node/org.knime.ext.h2o.spark.node.context.create.H2OSparkCreateContextNodeFactory)**
>
> Creates a new Sparkling Water context.

---

<div class="post-metadata">

**Author:** ![system](https://forum-cdn.knime.com/uploads/default/original/1X/ab3ccf34482a0329361734a18199390177204f15.png) [@system](https://forum.knime.com/u/system)\
**Post date:** [June 2, 2023, 9:01pm UTC](https://forum.knime.com/t/spark-pmml-model-predictor/15191/3 "2023-06-02T21:01:07Z")

</div>

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.
