I am completely new to KNIME and I will be honest I had no idea of its existence until a couple of weeks ago. So far I have thoroughly enjoyed the playing around with it and can’t wait to learn more.
Suffice to say I have encountered my first problem. I have a folder with about 50 pipe delimited text files, following some example I have been using the following:
Everything works great until it comes to the File Reader when using the Loop Node. I get the message:
“Execute failed: Too many data elements (line: 1 (Row0), source:……”
I am suspecting that this may be related to a slight change in the file spec where two additional columns were added. Is this what would have caused this?
I would appreciate some guidance on how best to deal with this as I need to find some kind of workaround to cope with these scenarios.
Hi @Page0727, welcome to the forum. It’s good to hear you’re enjoying your KNIME journey so far.
I am suspecting that this may be related to a slight change in the file spec where two additional columns were added. Is this what would have caused this?
It’s quite difficult to say without seeing the actual data and node settings. If you could provide these, that would be helpful so that readers here can re-create the error and do some troubleshooting. It would also be helpful if you let us know what exactly it is you’re trying to accomplish.
I can provide the workflow as shown above but the data is going to be a little more difficult as it is healthcare data. I am seeing that when I remove the files with slightly different spec it runs perfectly.
Now can I split the file after the list reader based on some identifiers that are in the file name RCNI9 or RCNI0.
The aim here is to pull all monthly files into one and then identify those that fall within dating criteria from the Month of the file back and whether they fall within the prior months. Thus identifying New or existing.
You are correct. This error is because File Reader expects same specification in every loop iteration. You can check node description and you’ll find a note about it and suggestion to use CSV Reader in this (loop) case. However if you are using newest KNIME version, 4.2.0, I would recommend you give it a go with new CSV reader node - CSV Reader (Labs). With it you can read multiple files that differ in specification without a loop. If you’ll be having trouble making it work drop me a message and I can create workflow example for you.
