Hello, I would like to know a way to transform the files that are inside one of folder in a once file to filter the data and have a unique file with all exclusive file with all information contend in all files.
For exemple, read the first file and save, read the second one, identify what information is different compared with the first and save in this first file only exclusive information and exclude the similiar information
Hi @tiagocasfer and welcome to the Knime Community.
You are not giving us much details. For example, what’s the type of the files? Structure? Do all the files have the same structure?
It’s usually best, along with your explanation, to give some sample data and what the expected results of that sample data should be.
In general, to compare tables, you can use these 2 nodes:
To find records that are in a table but not in the other table, you can use the Joiner node by doing either a Left or Right join
2 Likes
Hello Bruno Thank you for explanation. I will Try to get more detais!
I have one folder with somes files .TXT extension. This files are extrated everyday and can be countain the same informations that was saved yestarday ou can be have some new information.
Every night the extract program saved the file with a new name, increase one in the sume with the last file.
Like the example below
The first step is to find a way to get all files included in this folder and transform this informations in one file that contain all exclusive information .
csv reader (Mode from folder) can read you all your files in bulk
br
1 Like
Hi @tiagocasfer , it appears that you are using an old version of Knime.
The new CSV Reader allows you to read all the files at once from a given folder, and it merges the data from all the files for you.
This is how the new CSV Reader looks like with the new features:
As you can see, there is a “Files in folder” mode, and you can even apply filters on which files you want if ever you don’t want all files.
Are you able to update your Knime? If you can, then you can use this feature, and then use the Duplicate Row Filter to remove duplicates (that is if the same data from yesterday exists).
If not, we can look at alternatives.
Please let us know if you can update your Knime. The solution will vary depending on your Knime.
2 Likes
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.