Consolidated data have duplicate items

This is my workflow to consolidate the different files with the same information for one tab.

However, upon checking the output, seems that data consolidated were duplicated. See row 0#0 below.

See row0#3 below, same information:

Is there a question here?

my question is, is there something wrong on my configuration since the data is duplicating?

Hi @TLKNW , I see that this is using the workflow from here:

The workflow I supplied for that question was this:

but (as noted in that original thread and the above image) this was based on the assumption that the upper Excel Reader was reading all the files from the folder without a header, and then a single read (without data rows) from the lower Excel Reader was used to pick up a “good” header, to be applied to the resultant output.

However, you now have this within a loop based on List Files/Folders, so firstly you wouldn’t want to put the single Excel “header” read within the loop, and secondly if you are looping through all files and reading all files on each loop iteration, then yes you would have duplicates.

So… do you need the loop? (I don’t see what purpose it serves), but if you do:

  1. What is the config now on the upper Excel Reader?
  2. The lower Excel Reader (plus all the associated "Column Header and Transpose nodes) want to occur after the loop

Looks like you always read the same sheet?
is your excel reader configured correctly regarding the variables?

This is the upper excel reader settings.

And this is the lower excel ready settings.

I put a loop to copied all the data from the 30 files. Do I need to removed the loop to avoid duplicate data?

In addition, all the 30 files have the same header name, however I am not sure how to do the settings in order to consolidate it.

Update, I removed the files folder and the loops and it is now working perfectly.


This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.