Read more txt files with looping

KNIME Analytics Platform
, ,
1

Hi all,

I am reading with file reader more files in loop.
It may occur that one or more files from do(es) not exist at the moment, when the workflow try to reach a given file, because of that the workflow will be failed/terminated.

Is it possible to skip to the next iteration in case of fail of the loop?

Or

Should I use a scripting solution with a statement (if the file does not exist skip to the next iteration)?

How can I write this script in Python/R/Java?

Thank you,
Roland

2

Hi @rolandnemeth , I think to assist with answering this question, it would help us to know what the current mechanism you are using for performing the loop.

How do you determine the list of files if some of the files don’t yet exist?

Presumably you are not using something like the “List Files/Folders” node to generate your list, as then the files would all exist, unless something else is deleting them during processing?

Do you have a simple example of your workflow that you could upload, so that somebody can give some advice?

2 Likes
3

Hi @takbb,

Thank for your answer.
Please find attached the workflow.

Roland

Xray_CT_v2.knar.knwf (66.0 KB)

4

Hi @rolandnemeth , thank you for uploading your workflow. Obviously I cannot run the parts that connect to the database, and I have modified the list of files using some test files I have added to the workflow data area, but hopefully this gives an idea for a possible course of action.

I’m assuming that the data files you read are all of the same format so that they can be processed uniformly by the workflow.

image
image734Ă—215 12.2 KB

So what I have done here is placed your File Reader inside a Try-Catch. The lower port of the Catch Errors node takes an input to be used when the File Reader attempts to open a file that doesn’t exist. In this case, my Table Creator returns a dummy table with no rows.

For demo purposes I have supplied a set of CSV files with headings Name, Age.

I modified the initial Table Creator to point to my sample files:
image
In the workflow’s data area, though, I have supplied only the following files:
image

When it runs, it cannot find Test3.csv or TestX.csv, and in each of these cases it continues by processing the “empty table” supplied by the lower data port on the Catch Errors node.

I hope that gives a possible idea for how to make your flow work.

For the purposes of finding the demo files I included, I have also added a “convenience” component to the sample workflow.
image
Execute this to get quick access to the workflow data folder. This component is not necessary for the running of the workflow, but it just makes it easy to find the data folder

Xray_CT_v2 -takbb.knwf (121.4 KB)

(edit: Depending on what you need to do in the loop following the File Reader, you could also add an Empty Table switch downstream of the Catch Errors node, so that you only perform that part of the processing in the event of having a non-empty table from the Try-Catch block)

5 Likes
5

@rolandnemeth you could take a look at this example that uses the try and catch nodes with variables:

4 Likes
6

Hi @takbb

As usual great contribution. Moreover a great component added to the workflow as “extra present”, so convenient and useful to quickly/easily access workflow data :smiley: :+1: !

Thanks @takbb :clap: :smiley:

Ael

2 Likes
8

Thanks @takbb and @mlauber71,

mean time I created the following code, it does exactly the same.

Roland
R_Code_for_concat_multiple_textfiles másolat.docx (3.9 KB)

1 Like
9

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.