WARN Table Reader 2:4 Input file ‘C:\Program Files\KNIME\0’ does not exist
Node 2: List Files see the files (all 6) and lists them
Node 3 I’m assuming starts the loop
Node 4 should read the tables. I set Use Variable, currentIteration, under the Variable Settings???
Node 5 again assuming it will loop until there are no more files
Edit: I have checked it briefly and think adding example files into workflow directory and using workflow relative paths would be very nice. That way examples could be executed immediately.
Just seconding what Ivan already stated - nice work @TardisPilot! You should definitely upload your workflow to the Hub if you have time, for the widest possible availability.
Thanks! I will work on that and update the workflow then upload it to KNIME Hub. I have a few other examples (and some other great components built to auto load data from a users downloads folder etc.) that I want to include as well.
Hi - I can get the files into the file list and see that they are listed there. But when I want to add the variable, url is not an option. The only choice is not where the files are. How do get it to to show url as a choice so I can add it? If it isn’t done in the list files node, please specify what node you put this variable in. right now, I cannot point any of the nodes to anything except knime.workspace. Thanks.
I am not 100% sure what the question is. If flow variables do not show up immediately you could execute up to that point and try again or restart KNIME and start again. Or you could check with right click and the results of the previous node if the flow variables are there.
If you need more information on the location you could extract them.
And then it might make sense to open a new topic in the forum to asks your question.
From what I can see you don’t need to setup a variable at the List Files point since you’re telling the node to list the files from the folder that have the .txt extension. You will want to ensure that the Table Row to Variable Loop Start is just reading in the one file and that the file reader is set to the proper variable. Otherwise as long as those text files are the same shape you should be fine to loop over them.
@Tardispilot is correct here - you should apply the URL variable in the File Reader node, not the List Files node. (The URL variable doesn’t exist until the loop starts.)
Hi there, I suppose my question relates to this thread.
I am reading multiple (quite big by structure, 500K rows (roughly) and 158 columns (should be exact)) tsv (tab-separated values) files into one csv/excel. However, due to data quality (upon data recording) it happened that some files got less columns, i.e. 146 or 150. As I’m reading about 70 tsv files, it is quite challenging to go through each file and check if they are having the same number of columns or not. Moreover, as I’m processing each file (within the file reading loop), manipulating the data and transforming to a completely new table, it takes quite a long time. When such files with less columns encounter, the loop gives an error (for instance: “Too few data elements (line: 2 (File-3-Row0), source: ‘file:/C:/Users/KNIME/2W1P1.tsv’”).
I know that Concatenate Node takes care of different structured tables and concatenates tables even if they have different number of columns.
I thought the below settings would take care of the issue the same way the Concatenate Node does. But it seems not to be the case.
Can anyone help with this problem?
Thank you in advance.