Hey Everyone, I am currently trying to build a KNIME workflow to extract Job Descriptions from Webpages. This works well for most webpages, however, when I try extracting the description from any of the job postings on glassdoor it just results in an empty column and looking at the Xpath node it appears that the job description information is not in it (and not more than 2000lines). Here is an examplary link: [image] Jobs für Gesundheitsberater in Hamburg | Glassdoor Nach Gesundheitsberater-Jobs in Hamburg mit Bewertungen und Gehältern suchen. 1 Jobs für Gesundheitsberater in Hamburg. I have done this using the selenium nodes which appears to work. My questions now is: Where can I use the HTTP Retriever and where do I need to use Selenium nodes to capture content off a webpage? Thank you for your help! :slight_smile:

Please check here: [image] Scraping an AngularJS Website Post Load KNIME Analytics Platform Thanks for the pointer @ScottF ! @mpfeifer14 The HTTP Retriever (or the GET Request node) or any other techniques will just download the static HTML will thus not be of any help, as AngularJS requires a JavaScript environment, which is just available in a JavaScript-capable web browser. Without that, you’ll see the website as shown in an ancient browser without JavaScript (which means, placeholders instead of the actual content). So at the end this is not a question of the right XPaths, but the … – Philipp

Http Retriever cannot find job description

Community Extensions

qqilihq July 31, 2020, 7:20am 2

Please check here:

– Philipp

1 Like