Catch file reader error and use dummy file instead

Hi everyone,

I’m creating a workflow where I get files in a different folder every day. I modify the location of the file dynamically and use it as a flow variable for the node “File Reader”. But sometimes, files come with different types or columns, therefore I want to avoid getting an error in the workflow and use a dummy file to continue, and send an email if this happens.

How can I achieve this with Try/Catch error nodes? I seem to not find exactly how they would work in this scenario.

In short, if no error, the data should come from the dynamic location. If error, it should use the dummy file’s data and send email.

By reading another post from 2016, it seems the file reader (and excel reader too) fail if the configuration is incorrect but is not caught by the try/catch. They resolved it by checking if the file exists, but what if the file exists but has a different format? What is the correct way of catching that configuration error?

@ijoel92

You could check if a column is present and use an if switch accordingly

If the data is just empty you could use the empty table switch.

If you have a more complicated scenario you could take inspiration from the no 550 from this example:

And then you could use a try/switch construct for if the import fails. Demonstrated here for a database scenario but could be adapted to imports.

1 Like

Thanks for all the links. They were very useful to learn new things but unfortunately, they didn’t match my use case (I didn’t try R or Java nodes as I don’t know those languages).

If the file I’m expecting has new columns, the “File Reader” fails but is not caught by the Try/Catch and that is extremely FRUSTRATING. Maybe this feature should be taken into account in the future.

My workaround came from this answer which suggested to read the csv without a delimiter (none), and then use the “Cell Splitter” to generate all the data. This had an extra advantage (depending on how you see it) to cast all the data as strings.

There was an issue with this approach:
I change the name of the file dynamically to point to today’s folder through a flow variable. When there is no file, I point it to a file with the same columns and just one row of dummy values. Then, some nodes do the cleaning (filtering, reordering, renaming, etc). The problem shows when the file is not there, eventhough I connected to the dummy file which has the same format. Some nodes showed this error and it didn’t let the flow continue:
“Column range filter: Input table doesn’t contain specified column name”

(this happened on datasets that had a column named “Date” which was recognized as a different column after reconnecting. Bug?)

After A LOT of trial and error, I managed to get past it by adding a “Domain Calculator” and a “Table Validator” node after getting the data.

Sorry for the long text, I hope this serves someone if they find a similar issue!

3 Likes

Hello @ijoel92,

glad you found a way. On what version are you on? I have just tried File Reader with 4.2.1. using flow variable to control input and got error during execution.

Br,
Ivan

Sorry for the late reply, I just saw the notification.

I’m using version 4.2.0.

The string I use to connect to the file reader looks something like this (I’m using Windows):
“file:/D:/ETL%20Files/Affiliate%20Automation/ExportWPF/bin/downloads/empty%20dummy%20files/myfile.csv”

Hello @ijoel92,

hmmm… Not sure, as said I got error during execution so Try/Catch should work as expected. If you create an workflow example where Try/Catch doesn’t work properly I can take a look.

Br,
Ivan

1 Like

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.