I am scraping the number of fields of tennis clubs. In order to do so, I have a list with URL’s, which are used in the Start Webdrive node. Unfortunaly, I could not find any information on the forum with respect to the following error message that I get:
ERROR Start WebDriver 0:151:94:40:9 Execute failed: webDriver must not be null
The error comes up randomly and when I re-execute the Loop End node, the loop just continues where it stopped. Therefore, I conclude that it is not a malfunctioning URL.
Is someone maybe familiar with this failure? Thank you in advance for your reply!
I had a look at the workflow sent via email, yet, let me answer this here, as it is of general interest I think.
We will still need to figure out why the original webDriver must not be null happens after a few loop iterations. I currently only have a few assumptions, but we’ll need to dig deeper.
Generally, I advice against starting a new browser instance within loop iteration. This will cause quite some overhead for the start up time, and thus the workflow will run longer as necessary. Instead I suggest to start the browser using Start WebDriveronce before the loop body. Within the loop, rather use a Navigate node which opens the desired URL (instead of the “Start WebDriver”).
This should fix your original issue. I will keep this thread updated once there’s a fix for (1).
Update: Today’s 4.8.2 release fixes the “webDriver must not be null” issue which was caused due to some concurrency issue. (thanks for spotting this, @sequoyah!)
Yet, my general recommendation of above applies (start the browser only once, before the loop).