Knime 4.0 crashes randomly when using Excel Writer/Appender nodes.

Knime 4.0 crashes often during excel write or append sheet operation (xlsx). Not able to pinpoint since same node sometimes goes thru and sometimes results in crash.

When it does crash, is there anything in your log files (in your workspace directory, .metadata/.log and .metadata/knime/knime.log) that looks like a stack trace?

1 Like

Here is the link to one of the log files.

https://drive.google.com/file/d/1pFjRTgKsHvxlsnkm3Ue4M_yObdCunNX5/view?usp=sharing

That link requires permission to view; could you perhaps ZIP it and attach it to this thread?

1 Like

Here is the attachement.

hs_err_pid21076.zip (90.5 KB)

This says that your JVM ran out of memory; how much memory is being specified to KNIME (the -Xmx argument in the knime.ini in the KNIME directory?) How much physical memory does your machine have?

1 Like

HI,
Thanks for the reply. I have 16 GB memory and allocated 14GB using -Xmx option. Same workflow used to run without any issues in Knime 3.7.2.

Wow - that is cuckoo-banana that you’re running out of memory at 14G. I don’t believe we’ve change the POI version between 3.7.2 and 4.0.0 (POI is the Apache open-source project which we use to read and write Excel files) but i will check to make sure in the morning.

Are you able to describe in more detail (or export and attach to this thread) your workflow? Do you have a back-of-the-envelope estimate of how many rows are getting written at any one time?

It is only about a few thousand rows. I feel it has something to do with the new memory optimizations or the new open source JVM used in Knime 4.0

Yes, i think you could be correct - looking at the dump down in POI-land, there is a stack frame of sun.nio.cs.UTF_8$Encoder::overflow. I don’t know what kind of data you’re working with - could you ZIP it up and attach it to the thread?

If not, could you download Java 8 from Oracle, and in your knime.ini under the -vm argument cite the Oracle Java 8 JVM instead of the one bundled with KNIME?

1 Like

I will try with the Oracle JVM first.

Ok - sounds good; please report back.

When you report back, could you also describe the data being written (i.e is it basically all what would fit in US-ASCII, or ISO-8859-1, or … ?)

I tried Oracle JVM and it worked fine. Surprisingly, when I switched back to default JVM, there is no problem. Now I am at a loss.

Are you using the “Autosize Columns” in the appender node? This has caused me issues before.

That is very vexing… :- / I mean, great that it’s working now - but from the angle of ‘a deterministic world’: meh. … and there was no change in the data between the various JVM changes?

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.