If I were to merge 12 sheets each containing roughly 65,000 rows and 47 Columns. When I tried to loop through all the sheet names using the excel reader node, it tends to fail. How would I go about to solve the problem?
It is not clear what specifically fail, what’s the err message? Also, did you use example?
That what I am using and it is simply never finishes.
Then try load 1 or 2 sheets to estimate time.
I am working on this with my colleague (shubhamss).
Error message:Failed to execute runnable (java.lang.OutOfMemoryError: Java heap space)
I tested with a small sample size, the workflow has no errors.
How can I increase the Java Heap Space for KNIME?
- In the KNIME installation directory there is a file called knime.ini (under Linux it might be .knime.ini; for MacOS : right click on KNIME.app, select “Show package contents”, go to “/Contents/Eclipse/” and you should find a Knime.ini). Open the file,
- Find the entry -Xmx1024m and change it to -Xmx4g or higher (for example).
- (Re)start KNIME.
you can try giving more GBs to KNIME but still the underlying library of the Apache POI project performance is limited.