how to separate different files

Hi,

I have a pickle.

To illustrate a problem I will use a simple example.

I want to read some files (let’s say they are csv. and they are many of them). They are located in one folder. Each file contains table. They usually have the same structure but not always. So for instance we have 2 or 3 different structures:

image

image

image

The data differs sometimes in column header naming (date vs yyyyy-mm-dd) or extra columns.
I would like to separate the files (tables) based on some criteria. I wanted to read all of them and have the finale result:

image

My approach was to use column “date” to filter files that have this column and that have not. Then do some cleaning.

but somehow I do not know how to do it. Yes I still get the result but like this:
image

But I wanted to before reading it (and combining together) organize the headers.

To sum it up I would like to filter files and combine that are the same structure then organize header (rename it so it was clean) and combine all of them together.

If someone knows the way to separate the same tables it would help a lot.
image

If you can handle doing the reading first, then the Column Merger node will let you combine pairs of columns, e.g. ‘Date’ and ‘yyyy-mm-dd’ columns.

Otherwise I guess you could try something with a try/catch node pair.

Steve

1 Like

Hi @89trunks

See this wf seperate_different_filesknwf.knwf (68.6 KB) . It compares the table structures for all files and identify the files (filenames) with the same structure.


gr. Hans

3 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.