I have a big data set that I'm running thorugh a Group loop start. For some reason its creating junk rows (Row ID's like Cleared_Rowx ) and missing data for all other columns, because if which my loop end is taking enormous amount of time to join back all tables and the whole process is staling:
Intermediate table looks somethign like below-
Row1 ID Count Other Col's...
Row 0 23 45 .....
Cleared_Row1 ? ? ?...
Cleared_Row2 ? ? ?...
Cleared_Row3 ? ? ?...
Rows 2-4 are not in the original table at all not sure where they are coming from. Has anyone noticed this before and any leads for resolution ??
Sound strange. Are you joining or oncatenating any data in your workflow? Also, is it just `Group Loop Start` - Workflow - 'Loop End`? Any chance, you can provide the workflow you are running (favourable a reduced version, which only depicts the problem)?
Yeah its a little strange. Good news is the junk rows dont show up in the loop end result.
I have attached the workflow for reference, if you execute the loop end and open up & keep your group loop Start table view open. You should see all the junk rows flash by(hence I could not screen shot them). If I "step loop execution" I could not find them either, but I'm pretty sure they are there and causing a join issue.
Is this because I could be using "Streaming Job manager", resetting nodes etc. ?
Appreciate if you can take a look.