As mentioned at Memory usage - Excel Cell Updater , this node is using a lot of memory to read the entire Excel file into memory, in cases of big files (e.g. 74MB), it will consume between 25-60GB depends on cleaning the memory between nodes (manual garbage collection).
The logic required is passing through 24 Excel worksheets, and updating cells, overhaul of 11k cells are updated.
We for now had to put the entire logic into Python and openpyxl in order to update this file in a reasonable manner (15min run and 10GB memory).
As opposed to using Excel cell updater, using 60GB memory , and working in a loop over the worksheets, for more than 4hr, as each iteration using Excel cell updater to write cells into the Excel file, will take 10min just to open the entire file and update the cells.
2 Likes
Hi @david_abekasis,
thank you for sharing your feedback regarding the Excel Cell Updater node. I will forward the feedback to our developers.
Cheers
Kathrin