Selenium node workflow / Page parsing help for error

I am trying to get data from the page in my workflow and I get the following error. After resetting, it will continue to run again. I’m trying to figure out why the error was happening and I want to solve the error. Please help me for the error. @qqilihq

image
image

image

Hi,

Actually I’m not even able to read the node’s name easily. (I guess it’s Execute JavaScript)
What’s the output of Find Elements node when you get the error? (maybe the page is not loaded perfectly at first and there is no element found.)
What are the preceding nodes?
Anyway if the error cannot be solved (low probability), you can alternatively use this method to re-execute the nodes automatically:

Best,
Armin

1 Like

Hi Armin; @armingrudd
Can you look at the workflow and help solve the problem.

KNIME_Georgia__project.knwf (303.8 KB)

What do you want to extract from that “url” column (tr element)?
The “title” column is almost the same as the “url” column (just the “tr” element is removed)

Hi Armin; @armingrudd
both text and link information
image

So what exactly the “Execute JavaScript” node is doing??
You can get the same output without using this node. No other modification is needed, just remove “Execute JavaScript” node. Actually I think this node does nothing in your workflow. (Look at your column filter nodes, you remove the “title” columns produced by execute JavaScript nodes right after their execution). The input for the “Extract Attribute” nodes is the “url” column which is produced by the “Find Elements” nodes.

Beside that, I think you can optimize your workflow by removing unnecessary nodes (like “Execute JavaScript” nodes) and using the current version of the nodes.

Pay attention to the output of the nodes and the input they are using.

Best,
Armin

And about the error you get:
It’s because of the page loading. It takes about 2 or 3 seconds (in each Metanode after the “Click” node) for the page to be loaded completely. You have to use a “Wait” node after the clicks (not between the Metanodes).

Best,
Armin

Thanks for the answers, I’ll try what you write.