Selenium node workflow / Page parsing help for error

[umutcankurt]

I am trying to get data from the page in my workflow and I get the following error. After resetting, it will continue to run again. I’m trying to figure out why the error was happening and I want to solve the error. Please help me for the error. @qqilihq

[armingrudd]

Hi,

Actually I’m not even able to read the node’s name easily. (I guess it’s Execute JavaScript)
What’s the output of Find Elements node when you get the error? (maybe the page is not loaded perfectly at first and there is no element found.)
What are the preceding nodes?
Anyway if the error cannot be solved (low probability), you can alternatively use this method to re-execute the nodes automatically:

Re-execute on error

Hi David, I think I have found a method to do what you need: [re-execute] In this example workflow I have a Get Request node which may produce an error while executing (for example when there is no connection if you check the option in the configuration window). I could loop over the nodes until they executed successfully and I got what I wanted (in this case a list of countries). I used a “Generic Loop Start” and “Try” before the main nodes (no configuration) and after the main nodes used…

Best,
Armin

[umutcankurt]

Hi Armin; @armingrudd
Can you look at the workflow and help solve the problem.

KNIME_Georgia__project.knwf (303.8 KB)

[armingrudd]

What do you want to extract from that “url” column (tr element)?
The “title” column is almost the same as the “url” column (just the “tr” element is removed)

[umutcankurt]

Hi Armin; @armingrudd
both text and link information

[armingrudd]

So what exactly the “Execute JavaScript” node is doing??
You can get the same output without using this node. No other modification is needed, just remove “Execute JavaScript” node. Actually I think this node does nothing in your workflow. (Look at your column filter nodes, you remove the “title” columns produced by execute JavaScript nodes right after their execution). The input for the “Extract Attribute” nodes is the “url” column which is produced by the “Find Elements” nodes.

Beside that, I think you can optimize your workflow by removing unnecessary nodes (like “Execute JavaScript” nodes) and using the current version of the nodes.

Pay attention to the output of the nodes and the input they are using.

Best,
Armin

[armingrudd]

And about the error you get:
It’s because of the page loading. It takes about 2 or 3 seconds (in each Metanode after the “Click” node) for the page to be loaded completely. You have to use a “Wait” node after the clicks (not between the Metanodes).

Best,
Armin

[umutcankurt]

Thanks for the answers, I’ll try what you write.