exception thrown

Node Development
1

I just got this error message (after a few hours of computing…, quite frustrating…) Is this an OS problem or a software problem?

Thanks,

B

java.io.FileNotFoundException: /tmp/BatchExecutorOutput12635684951960/Row Filter (#18)/port_0/data.zip (Too many open files)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.(FileOutputStream.java:179)
at java.io.FileOutputStream.(FileOutputStream.java:131)
at org.knime.core.data.container.ContainerTable.saveToFile(ContainerTable.java:179)
at org.knime.core.node.BufferedDataTable.save(BufferedDataTable.java:362)
at org.knime.core.node.NodePersistorVersion200.savePort(NodePersistorVersion200.java:258)
at org.knime.core.node.NodePersistorVersion200.savePorts(NodePersistorVersion200.java:179)
at org.knime.core.node.NodePersistorVersion200.save(NodePersistorVersion200.java:137)
at org.knime.core.node.workflow.SingleNodeContainerPersistorVersion200.save(SingleNodeContainerPersistorVersion200.java:212)
at org.knime.core.node.workflow.WorkflowPersistorVersion200.saveNodeContainer(WorkflowPersistorVersion200.java:516)
at org.knime.core.node.workflow.WorkflowPersistorVersion200.save(WorkflowPersistorVersion200.java:387)
at org.knime.core.node.workflow.WorkflowManager.save(WorkflowManager.java:4172)
at org.knime.core.node.workflow.BatchExecutor.mainRun(BatchExecutor.java:418)
at org.knime.product.headless.KNIMEBatchApplication.start(KNIMEBatchApplication.java:102)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:193)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:386)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:549)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:504)
at org.eclipse.equinox.launcher.Main.run(Main.java:1236)
at org.eclipse.equinox.launcher.Main.main(Main.java:1212)

2

This is a “too many open files” problem. There is limit of how many open files a process can have on linux (and probably also windows). The default on linux systems is 1024 as far as I know. Here is web page I found describing how to increase the limits.

How large is your workflow?

3

Not sure what you mean by large? how many node? or how many rows? It turns out that this particular workflow has 8 parallel tracks, each with roughly the same workflows including nested workflows on 2 sub-levels. There are probably about 11 million rows per node… We changed the value to 2048 and it works now… (so far, got into this mysql-heap problem now…)
B

4

Yes, I meant worklow size in terms of nodes (the row count shouldn’t matter match). But then again, I know users who have 3000+ nodes and 100+ meta nodes in their workflows and they run it without problems, so it really depends on the type of nodes and the data. Maybe we should have a look at your specific problem during the UGM.

Thanks!