Creating a loop to import data to BigQuery

Hi,

I am trying to create a workflow to import 1400 CSV files to Google BigQuery. I start with using Table Row to Variable loop start to get the URL of the files, and then File Reader to read each table. After data manipulation, I use DB Table Creator and then DB Loader but I cannot connect the Loop End node to these nodes. Am I using the wrong nodes or doing sth wrong?

Thank you!

Hi @datazoom and welcome to the forum.

You could try the Variable Loop End instead, it’s a bit more flexible since every node inherently has flow variable ports included already.

3 Likes

Thank you, It worked.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.