Excel slow writing(Appending)

Not sure if it’s recreating the file as new sheet append in. I do have a work flow that creates probably 10 sheets in total and the final file is around 40~MBs. Today as it got to appending the 7th or 8th sheet, it’s getting slower and slower till it froze my laptop. Is there anyway to solve this issue?

Thanks,
Eric

Look at Autosize columns option. If it checked it could slow down the process.

Hi Izaychik63,

Thanks for your response. I double checked and didn’t see the box checked. Any other tips?

Thanks,
Eric

Hi there,

this was one time or this is still happening? Which KNIME version are you using?

Br,
Ivan

Hi Ivan,

It’s still happening. I swapped to a stronger machine, but when it got to the last few files the same behavior started happening. Believe I have the latest 3.7.2 version.

Thanks,
Eric

Just wandering, if you start from problematic files behavior will be the same?

Hi Eric,

was this behavior from beginning or after some update or? What exactly are you doing - creating Excel file and then appending sheets? If you can share your workflow it would help to investigate what is going on…

Br,
Ivan

I put up a sample workflow and mock up data here. The size of the data is the same as the one I was working on. I have about 16 sheets being appended to the same excel file. It will get slower and slower when it gets to the last few files.

KNIME_excel_append.knwf (58.1 KB)

sample file:

So I did a test with upgraded VM with 8 core and 32 GBs memory. It went through and took about 15 mins. I noticed it took up 13 GBs of memory towards the end of the job. It’s still weird since the excel file is only 28~Mbs big.

Eric

Hi Eric,

I have checked it and got my laptop frozen couple of times :slight_smile: With every new sheet, appender node takes longer and seems the problem is the underlying library of the Apache POI project. It is noted in the node description of Read Excel Sheet Names (XLS) node. So I’m afraid upgrading your machine might be the best approach unless someone has any other idea…

Br,
Ivan

1 Like

Thanks for checking :wink:

Eric

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.