How to use a running number / identifier across multiple filed

Hallo everyone
I am a new user, so please excuse me if I ask stupid questions. I didn’t find this one by searching, so I hope you can help me.
I have a large file with thousands of entries that are identified with a unique identifier. Each month new Data sets get added in.
I have just programmed a way to aggregate the data from different CSV files.
How can I merge the data, and make sure that the running number/unique identifier is used continuously on the new data?

Its not very clear how your data is structured. You say “I have a large file with thousands of entries”. And then “I have just programmed a way to aggregate the data from different CSV files.” Does this mean you’re feeding the final “large file” from new monthly files? I assume the unique identifier appears in the monthly file. If so I don’t really understand your issue. Could you share some sample data and explain what you’ve done so far and why it doesn’t seem to work.

You can use the Counter Generation node as a quick way to assign a row count ID during processing. Not a Unique ID for a database, but I use it quite a bit in KNIME workflows.

Hello all. Apologies for being unclear, but thank you for the answers already!
I am essentially attempting data reconciliation between information from banks and a separate cash tracking app.
The banks send differently formatted csv files, which I have now read into my workflow, harmonized and aggregated. Let’s call this “Bank_Input_New_month.XLSX”. This was my first project ever in KNIME and I am extremely proud that it worked (thanks to the help from the forum here!).
Now comes my next problem:
Unfortunately the data from the bank is not from 1st until 31st of a given month, but from various dates - like from the 5th of Jan until the 4 of Feb. And every bank is different.
That is why I have one continuous file with all aggregated data - called “All_Bank_Input_Ever.XLSX” in which I collect all bookings and can compare it against the cash tracking data on a month-by-month basis (ignoring the data that spills over into the next month, until I have the data from that month complete).

In the All_Bank_Input_Ever file, I had so far always given a unique number to each dataset to be able to filter and sort, but be able to revert back to the original order of the dataset. I have not worked with this file in KNIME yet - this was all done manually.

My planned next step is to also read in the All_Bank_Input_Ever file into my workflow (or a new one - don’t care) and merge the new data from “Bank_Input_New_month.XLSX” into the file.
Merging the files is no problem with the Joiner node, but how can I make sure that the new datasets from Bank_Input_New_month.XLSX can not only get included into All_Bank_Input_Ever, but also get assigned the continuous numbers that the existing datasets already have?

I hope this clarifies a little better. I can try to create two dummy files tomorrow, if needed.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.