Append only new files in loop excel reader

[Waldemar]

Hello,

with the following loop i read several excel files from one folder and append them to one big table in knime.

Currently in the folder there are about 55 files (each with 90k rows). Each month i add one more file to the folder. The other files are not changing. It takes a lot of time until the loop is finished. My question is, is it possible that the loop only adds the data from the new file in the folder to the existing table? So that the loop does not have to run through everything again?

Your help is greatly appreciated.

Best Regards
Waldemar

[HansS]

Hi @Waldemar

Welcome to KNIME Forum. My suggestion:
You have a folder with excels e.g. C:\Data\tmp\tmp_excel and a subfolder done

With the EXCEL reader you can read in all (3) xlxs files in one go.

After you read in the files you can move them to done with the File Transfer node.
Your folder only contains the result file

When a new excel file arrives in the folder you can execute the flow again.

See this wf transfer_files.knwf (36.7 KB)

gr. Hans

[gonhaddock]

Hello @Waldemar , and welcome to the KNIME community.

I have a workflow, that you would find it useful for your use case.

File Management _ Read Incremental Files in Folder – gonhaddock

Let me know for clarifications, or further support if needed.

BR

PS.- This workflow needs to be revisited and updated… but the concept is clear.

[Waldemar]

Thanks for your solution. Helped me a lot

[Waldemar]

Thanks for your help:)