To be honest I do not fully grasp what you want to do. Maybe an example workflow that demonstrates your full challenge without giving away any secrets might alosnhelp.
Concerning duplicates, question is if the use of a rank Id might help like in this example:
Also the corresponding article and sample workflow could give you some inspiration, maybe.