I want to get reviews from a web page,
My workfow contains find element , click nodes. Firstly i started workflow. There was no problem. After a while ,
Web page started to slow . find elements node takes a long time . i want to take 50 000 reviews. İs there any solution for this?
qqilihq ? have you got any solution?
Sorry, without more details it's hard to provide any help. Can you supply your workflow?
i want to get 50 000 reviews but chrome doesnt load reviews. i can get 4480 reviews .
Maybe you can see much better, this wokflow works good. i used different browsers but i can get maximum 4480 reviews. Google play store reviews
Soner, please attach the actual KNIME workflow, so that I can open and try locally in KNIME. You can export it from KNIME as ZIP file through the "File" menu.
okey how can i send zip file to you?
Either use the "File attachments" function here in the forum, or send it to email@example.com
the issue is, that during scrolling the browser will simply accumulate and keep all shown reviews in the DOM. That's why after having processed several thousand reviews, the process will become very slow. I had a similar issue recently, which happened when trying to scrape all Tweets from a Twitter profile.
var element = arguments;
var parent = element.parentNode;
If that's of any help, I can sent you the workflow for Twitter. It will of course require some adaptation to your specific use case.
Thank you for for answer. i understood the problem.Can you sent me your twitter workflow .Maybe i can manage to solve for my workflow.İf i solve the problem, i will inform you.
yes, I'll send you the workflow in a second.