Looping with errors

[henryhcraig]

Hi!

I’m struggling with something that I feel should be very simple, but can’t find any guidance on it.

I have a web scraper which uses Selenium nodes to loop over different pages.

I want to dump the results into a table, so have used a Table Writer node after the loops.

For some pages however, there may be an error, and I want to save the data which was collected before the error. IE ie want it to write for every row, and “save” the intermediate results.

I can’t work out how to do this as if there is an error the workflow stops and no data is saved.

Any ideas?

[oole]

Hi @henryhcraig

You can use a Try/Catch inside the loop, so that only the successful retrievals are collected in the Loop End node.

You can find an example of such an approach in this post and provided workflow: Twitter get n-number of timeline from each user (loop)

In that particular case the looping happens over twitter handles, some of which cannot be accessed and are skipped.