I am currently trying to loop through a folder of text files and build an outcome table from each file resulting as a row. Each text file is pipe delimited and the file reader can interpret them as a single row with 13 columns. However, when I loop through all of the files, the file reader only has one row, and the concatenation node only has two lines. What am I doing wrong?
My work flow looks like:
List Files -> Table Row to Variable Loop Start -> File Reader -> Concatenate -> Variable Loop End
What are you concatenating with. This should be an unnecessary node.
all you should need is:
list files node
table row to variable loop start node
file reader node
loop end node.
note in the file reader node you need to go to the flow variables tab and select the list files variable from the drop down next to the filename section.
Thanks for the speedy response. For the record, I am using KNIME 2.7.1.
The issue that I have is that the File Reader Node loop, as you suggest, when completed only lists the last file read in the loop as a single row. I have tried to connect other nodes to the File Reader to see if all rows were being held in memory, but not displayed, but alas they also only show one row.
It appears that I need another node to collect the results from the file reader as it iterates through the files. That is why I assumed the Concatenate Node would work as it would add each row through a union to the same table. But, this is not working either.
I have attached a screenshot of the File Reader Node output.
Here you go. I have included the workflow as well as the fake data. In the data are four text files, each containing four pipe seperated elements. Note that there are no carriage returns in the text file becuase if present the file reader will want to interpret those as row indications and would return four rows with two columns, one empty.
Please let me know if you have any solutions. I have tried several permutations with concatenate and others but am getting no where fast.
If I understand correctly, Simon's suggestion of a standard loop end node is what you are looking for. Have a look at the attached example and let me know if this is indeed what you are trying to get.
Fantastic! Thank you. I had assumed that the nomenclature implied the neccessity of some loop ends being associated with some loop starts. I now understand their independence and use...thanks to you both for sticking with the question!
note in the file reader node you need to go to the flow variables tab and select the list files variable from the drop down next to the filename section.
I'm trying to do this but can't find this dropdown menu - has this changed?
If you want the entire file as a single table row, then you can dispense with the loop altogether, and use List Files --> 'Load text-based files' node pairing (the latter is from the Vernalis Community Plugin - Community->Vernalis->Miscellaneous->Text-based IO->Load text-based files