I am new to KNIME and I am using it for the first time. I used KNIME for comparing two data sheets of excel sheets. One sheet was a dump from the old software database and the other is the dump from the new software database. So simple is we taken dump from the old software in excel. Done manual mapping and then uploaded this database in new software. Now again took one dump from new software in excel. So now we are coming to this two excel for data accuracy in data migration.
So here I used excel reader to upload both excel sheets in KNIME and created two nodes for this, then used joiner to compare data for 6 to 7 columns so it can give data matching and any error it will not compare so can find our data migration accuracy between two data.
Here I found some issue is when joiner reaches at 99% it got freezes for a long time. Data size in each excel sheet would be 6 to 6.5 lacs (650000) rows. I have waited for around 2 hours but the joiner is not going further from 99%. Here I would like to add that previously i had used the same workflow for two excel sheets which was having some 10000-row data and that worked effectively. But this time joiner does not move further from 99%.
Please help me to resolve the issue. Also please suggest any better way of doing data comparison for data accuracy and quality of migration.