Java heap space

[ahmed_gomaa]

I increased the Java heap space to be Xmx8g but i still get the error “java.lang.OutOfMemoryError: Java heap space: failed reallocation of scalar replaced objects”.

Is there any fix?

[ana_ved]

Hi @ahmed_gomaa

Thank you for your question.

I saw you had another post about this before. Did you restart your KNIME? Also, depending on how heavy your data mining is, you could need a bit more of memory. Can you tell me which nodes throws this error?

[ahmed_gomaa]

Yes, I have a previous thread but I can’t reply on it. Yes, I restarted the KNIME. The server’s memory is 12 GB. I stuck with the Keygraph Keyword Extractor node. I have a massive document data set.

[ahmed_gomaa]

After partitioning, the data set size about 10 million records.

[ana_ved]

Hi @ahmed_gomaa

Have you checked if the problem persists if you use a small subset of your data (i.e., 1000 records)?

If your dataset is massive, possible solutions I could think of would be to keep increasing your memory or preprocess your data a bit more to get a more concise dataset with just the most relevant information for you. Maybe check if you have duplicate records, redudant information, if you can remove some stopwords, and so on.

Also, if you use some nodes that can use the streaming mode, it is worth to make parts of your pipeline using it. This would speed up your general workflow and help with memory problems. The link I have attached contains more information on how to use streaming.

I hope this helps a bit, let me know if it doesn’t!

Cheers,
Ana