How to upload a huge Excel file (500MB)

KNIME Extensions
1

Hello all,

I am a beginner and I would like to ask you if there is a way to upload a big data set in Excel file (500MB) through Excel Reader node. The file includes generated vectors from text and their values (I'm learning text processing). I need to use XLSX format. Currently, I am not able to upload the document due to its size (it takes ages). Is there a way to tackle this problem? 

Thank you!

Lena

2

Hi Lena,

have you tried increasing the memory that KNIME can use in the knime.ini file? You can do that be changing the -Xmx parameter in this file e.g. to 8GB (if possible on your machine).

The second thing that I would try is to avoid XLSX and convert the file into a CSV file and use the File Reader node or the CSV Reader node. Parsing huge XLSX files is quite memory intense.

I hope this helps.

Cheers, Kilian

3

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.