Scrolling with Selenium's Execute Javascript node

[Chris_88]

Hello everybody,

I have a problem with the execute javascript node in my workflow to retrieve informations on the Dimensions website.

To get more results, it move the cursor down to load a new page of results.

To do this, I use the following command line in Selenium’s execute javascript node :
window.scrollTo(0, document.body.scrollHeight);

This command loads several pages in a row which makes it difficult and slow to exploit html code by findelement node.
Is there any way to modify this command to load page by page?
KNIME_project7.knwf (48.5 KB)
Thank you for your help.

Best regards,

Chris

[armingrudd]

Hi @Chris_88,

Maybe you wanna try the focus() function:

Selenium Web Page help scrolling down

Hi, That’s an infinite scroll and you can expand the result sets by applying this code in “Execute JavaScript” node: function getElementByXpath(path) { return document.evaluate(path, document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue; } getElementByXpath("/html/body/div[2]/rhr-search/rhr-main-content/main/ng-transclude/div/div/rhr-search-result/div/div/ol/li[20]/rhr-search-result-row/h3/a").focus() Just remember that this code focuses to the 20th result which is t…

Or if it doesn’t work, find the last element using the Find Elements node and run this code in the Execute JavaScript node while checking the last element in the arguments list.
arguments[0].scrollIntoView();

[Chris_88]

Hi armingrudd,

Thanks for your help.

Best regards,

Chris