R snippet error

I use a worked R snippet on a massive data set. I get this error “cols[[1]]<-integer(rowCount) knie error”. The column number is changed if I try to re-execute the node. Any suggest?

Hi @ahmed_gomaa -

I’m afraid we’ll need more information to suss this out. Do you have a workflow you can post, along with some sample data? And perhaps a portion of the log?

When you say “massive data set”, what do you mean - very wide, very tall, or both? Roughly how large?

Since the column number in the error message is changing when you re-execute the node, this might indicate you’re running out of system resources (but this is only speculation on my part)…

The data set consists of 89 columns and over one billion records.

The Log error

2019-09-24 11:48:46,764 : ERROR : KNIME-Worker-30 : : Node : R Snippet : 0:18 : Execute failed: R evaluation failed.: “cols[[24]]<-factor(as.character(rep(NA, rowCount)))”
org.knime.r.controller.IRController$RException: R evaluation failed.: “cols[[24]]<-factor(as.character(rep(NA, rowCount)))”
at org.knime.r.controller.RController.eval(RController.java:441)
at org.knime.r.controller.RController.monitoredAssign(RController.java:1131)
at org.knime.r.controller.RController.importDataFromPorts(RController.java:528)
at org.knime.r.RSnippetNodeModel.executeSnippet(RSnippetNodeModel.java:201)
at org.knime.r.RSnippetNodeModel.executeSnippet(RSnippetNodeModel.java:170)
at org.knime.r.RSnippetNodeModel.executeInternal(RSnippetNodeModel.java:149)
at org.knime.r.RSnippetNodeModel.execute(RSnippetNodeModel.java:136)
at org.knime.core.node.NodeModel.executeModel(NodeModel.java:567)
at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1192)
at org.knime.core.node.Node.execute(Node.java:979)
at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:559)
at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)
at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:179)
at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:110)
at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:328)
at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:204)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)
Caused by: org.rosuda.REngine.Rserve.RserveException: voidEval failed, request status: error code: 127
at org.rosuda.REngine.Rserve.RConnection.voidEval(RConnection.java:182)
at org.knime.r.controller.RController.eval(RController.java:436)

Hi @ahmed_gomaa -

Do you see the same error if you use a much smaller subset of data, or only on the large dataset?

If it’s the first case, posting your workflow (including the R code) would be useful.

If it’s the second case, maybe increasing memory available to KNIME would help? More info on how to do that is available in our FAQ.

You could provide us with more details about your environment. Eg running these comands in your R or RStudio or in KNIME and see the Excel File. My first guess would be problems with performance.

R.Version()
Sys.getenv(‘JRE_PATH’)
Sys.getenv(‘MANUFACTURER’)
Sys.getenv(‘MODEL’)
Sys.getenv(‘MSYS2_ENV_CONV_EXCL’)
Sys.getenv(‘NUMBER_OF_PROCESSORS’)
Sys.getenv(‘OS’)
Sys.getenv(‘OS_LanguagePack’)
Sys.getenv(‘OS_VER’)
Sys.getenv(‘PROCESSOR_ARCHITECTURE’)
Sys.getenv(‘PROCESSOR_IDENTIFIER’)
Sys.getenv(‘PROCESSOR_LEVEL’)
Sys.getenv(‘R_ARCH’)
Sys.getenv(‘R_COMPILED_BY’)

kn_example_r_environment.knwf (8.4 KB)

2 Likes

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.