Please i want to know how to read large excel files. I trying with a read node but stop working after while.
What can i do?
Hi Gustavo,
which read node are you using? Did you already try the "XLSX Reader"? For big files it may take a while to load the data.
Best,
Christian
In those cases I prefer to convert the Excel file into CSV, but before you should remove in Excel any special caracters like "," or ";" that can affect the column separation and also replace the page breaks by CTRL+J by nothing.
I prefer to use the File Reader node than the CSV Reader.
Hope it helps
1 Like
Just as alternative: besides switching to CSV, you can treat huge CSV files as tables with Hive; Hadoop will do the "import" work for you. From there on, you can query the huge file just as it was a DB table.
This is a different avenue, but it scales very well.
2 Likes