How to scraping 20K website address (www) to find the some keys?

KNIME Analytics Platform
#1

I wonder if anyone knows how to scrape 20K website address from the list and marked the list consist of keys ( for instance ( bad word OR sexual assault key ) ?

BR,
Milad

#2

I normally use phyton script.
Regards

#3

Dear Mola,
Thanks for your reply.I mean scraping method to check the context of the index and some main pages/

BR,
Milad

#4

ah Ok, you can try this web scraping

2 Likes
#5

or this How can I get the tips and tricks of the welcome page altogether?

2 Likes
#6

Hi @natanzi,

Do you have a few example pages you want to scrap?
If it is a simple get+parse step the Get Request Node might be enough

Combined with a XPath Node

For more complex webscraping you can have a look into the selenium extentions:

2 Likes
#7

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.