append method in the write table node

[Fabien_Couprie]

to add an append method in the write table node as it is in the csv writer ?

[nan]

Hi @Fabien_Couprie ,

please consider to join forces and vote for this older feedback thread: Table Writer -- Append if file exists

Thank you for your feedback, it totally makes sense, we simply didn’t get to it yet.

nan

[mlauber71]

@Fabien_Couprie I am sure you are aware of this. You can use parquet writer to get this function.

KNIME Snippets (1): Collect and Restore — or how to handle many large files...

[Fabien_Couprie]

I tried and had some reliability problems in the past. The fact is that in 5.7 I have this interface :

[mlauber71]

@Fabien_Couprie the secret is in using the Folder setting with the Parquet Writer

Parquet - Split Data into several Files – mlauber71

Problem reading/writing big file on hub space

@tttpham the idea is to write the large file out in several parquet files that you can then upload individually to the new destination [image] There a Parquet reader will bring the data back into KNIME and handle them as a single file: [image] You can also make sure they have the same structure: [image]