csv file content changes when using a loop

Hello,

I tried to read in a list of files, perform operations, and output the files. However, I found the data content of the csv files changes even if I don’t have any node in between the csv reader and writer node(no data operations). This happens only when the number of files gets larger by looking at the first and the last data in a certain cluster. For example, the data in the last cluster, cluster 32 changes. The work flow is attached. Please provide guidance. Thanks.
output csv change problem.knwf (137.7 KB)

Regards,
Jack

Hi @jackcao53

I think the issue is you are creating the new file name from the loop iteration and not from the name of the original file. Does that make sense? The last line in your file list is cluster 9 but in the loop it’s iteration 32 and you are writing a file with that name. So you put the data from cluster 9 into a file named 32.

Let me know if this helps,
Thanks,
Jason

1 Like

Hello,

Thanks for your reply. I don’t know why it shows like that, but I actually got 33 output files from 33 input files, and the first few files actually match. The last few files don’t match (data changed). I didn’t check the files in the middle. Maybe I should try to use the name of the original file? How can I do that? Thanks.

Regards,
Jack

@j_ochoada forgot to @ you.

Hi @j_ochoada,

I changed the new file location to according to original file name, and the problem disappeared. Thank you very much!

2 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.