I’m creating a workflow where I get files in a different folder every day. I modify the location of the file dynamically and use it as a flow variable for the node “File Reader”. But sometimes, files come with different types or columns, therefore I want to avoid getting an error in the workflow and use a dummy file to continue, and send an email if this happens.
How can I achieve this with Try/Catch error nodes? I seem to not find exactly how they would work in this scenario.
In short, if no error, the data should come from the dynamic location. If error, it should use the dummy file’s data and send email.
By reading another post from 2016, it seems the file reader (and excel reader too) fail if the configuration is incorrect but is not caught by the try/catch. They resolved it by checking if the file exists, but what if the file exists but has a different format? What is the correct way of catching that configuration error?
Thanks for all the links. They were very useful to learn new things but unfortunately, they didn’t match my use case (I didn’t try R or Java nodes as I don’t know those languages).
If the file I’m expecting has new columns, the “File Reader” fails but is not caught by the Try/Catch and that is extremely FRUSTRATING. Maybe this feature should be taken into account in the future.
My workaround came from this answer which suggested to read the csv without a delimiter (none), and then use the “Cell Splitter” to generate all the data. This had an extra advantage (depending on how you see it) to cast all the data as strings.
There was an issue with this approach:
I change the name of the file dynamically to point to today’s folder through a flow variable. When there is no file, I point it to a file with the same columns and just one row of dummy values. Then, some nodes do the cleaning (filtering, reordering, renaming, etc). The problem shows when the file is not there, eventhough I connected to the dummy file which has the same format. Some nodes showed this error and it didn’t let the flow continue:
“Column range filter: Input table doesn’t contain specified column name”
(this happened on datasets that had a column named “Date” which was recognized as a different column after reconnecting. Bug?)
After A LOT of trial and error, I managed to get past it by adding a “Domain Calculator” and a “Table Validator” node after getting the data.
Sorry for the long text, I hope this serves someone if they find a similar issue!