table row to variable to read multiple CVS, process and append in 1 file

Hi, it should be simple but it seems I cannot get it to work:
I have a list of CVS files that I need to process (extract 3 rows, I use the row splitter) and then write the result in a final CVS where the rows from the files are appended (row 123 of file 1 then 123 of file 2…). See below the simple workflow. The problem is that it does iterate over the number of files but write and append always the same 3 rows coming from the first CVS.

It would be better if you posted a downloadable version of your workflow so that we could peek under the hood.

My guess just from the picture and description is that the CSV reader isn’t configured to use the appropriate flow variable. Can you show us the result of the Table Row to Variable Loop Start as well as the configuration for the CSV Reader node?

It’s also unclear to me how the row appending is supposed to happen, since every loop goes directly to the CSV Writer node.

Also, why are you using deprecated nodes?

1 Like

Hi, Thank you so much for taking a look. You are most likely right. This is the all workflow. I would add a couple of my CVS but it is not apparently an accepted format.
John_TEAD.knwf (109.3 KB)

1 Like

I’m using deprecated nodes because I am trying to built something from workflows I have found and they have deprecated nodes… sorry, I’m very limited in my KNIME knowledge

No problem. Are you working in KNIME 4.3? This version has some differences compared to previous versions that will affect the solutions I suggest.

Correct it is the version 4.3.1

Ok, great.

When I opened the workflow you provided, the CSV Reader node was not configured to accept a flow variable for the file path. It requires a Path type flow variable, and you’ll need to create this.

The updated List Files/Folders node will automatically create this Path data type. You won’t need the String Manipulation node.

Replace the deprecated Table Row to Variable Loop Start node with an updated version. This new node will be compatible with the Path variable we need to produce. Once replaced, you can run it and check the output. The variable we’re interested will be called “Path” and the icon will be a “P”.

Now to configure the CSV reader node. Make sure that the Mode is set to “File”, then click the button to the right of the Browse button, check the box marked “Use Variable” in the window that pops up, and choose “Path”.

Once done successfully, you’ll see a reminder at the bottom of the window that says that the “path” parameter is controlled by a variable.

Next, we need to compile all these rows. In KNIME this is called row-wise concatenation, and the Loop End node does it quite nicely. Therefore, I’d add a Loop End node immediately after the Row Splitter node and then follow that with the CSV Writer node.

image

I tested this arrangement with some of my own CSV files and it seems to work the way it should.

Give it a short and we’ll go from there. If the incoming CSV files don’t all have the exact same column names, we’ll have to make look at some extra configuration options in some of the nodes.

2 Likes

It works beautifully!!! thank you so much!

the problem is that I need to add a column with the number at the end of the file title in each row (to match the row with the file) and I with the other workflow I extracted with String manipulation from the URL or location but with the new nodes Path (that contains the number) is not available (I guess because it is a variable already). Any suggestions to extract the 10008488 (see below) ?


Thanks

You can try to use a Path to String (Variable) node to convert the Path variable to a string, then process it like you did previously.

That is indeed a great way to do it, all set! Thanks

2 Likes

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.