Scrolling with Selenium Nodes?

[am_dbs]

Hi Philipp,

I've (again) an issue concerning the Selenium Nodes.

I've a website (xing) which updates itself (respectively adds content) when I scroll down - so when I try to extract content using the Selenium Nodes (Find Elements) only the first hits can be extracted. Is it possible to scroll to the end of the page, using the Selenium Nodes? Or do you may have other suggestions?

Many thanks in advance!

Best
Simon

 

 

[qqilihq]

There are currently no dedicated nodes for scrolling, however you can use the "Execute JavaScript" node. Have a look at the Facebook workflow from the Selenium examples, it does exactly the same.

Best,
Philipp

[am_dbs]

Hi Philipp,

I've had a look at the scrolling-process of the selenium-facebook example - unfortunately this does not work for the xing-page.

When I add the loop (with 20 iterations) including the javascript-node (window.document.body.scrollTop = 500000000000;) and the wait-node and execute the workflow - the xing-page does not move during the workflow-process. As result only the first 23 hits (of ~170) were extracted.

Is there maybe any other option to scroll?
Many thanks in advance!
Best
Simon

[qqilihq]

Hi,

I've just verified (using Safari) and scrolling on XING works for me using the given JS snippet. But you might alternatively give something like ..:

window.scrollTo(0, document.body.scrollHeight);

... a try, which scrolls to the bottom of the page. Alternatively, you can explicitely scroll to a Web Element, by passing it to the JavaScript node as argument and doing the following:

// first argument passed in arguments list
var element = arguments[0]; 
element.scrollIntoView(true);

Kind regards,
Philipp