Webpage Retriever - enable JavaScript to run this app

[Ardilla]

I’m trying to use Webpage Retriever to extract information about flights in Skyscanner; however, I’m getting " You need to enable JavaScript to run this app." as part of the result in the body section

Am I missing a configuration in the node to resolve this issue?

[qqilihq]

This node will not help in this case - you can have a look here for an explanation and suggestions:

Scraping an AngularJS Website Post Load

Hello Everyone, Was wondering if someone could help. I am trying to scrape an angularJS website. When using the HTTPRetriever it only scrapes the HTML without the variables names written in/still has only placeholders. Was wondering if anyone knows how to work around this. Is there a way to allow for the page to fully load? A special trick to retrieving xpaths for Angular websites? Preferably without having to use a selenium node but that might be overly picky. Thanks in advance!

HTML Parser Problem

Hi Ham, Good to hear that it worked (somewhat ) - the discrepancy between the DOM view in the browser’s dev tools and the XPath node comes from the fact, that the content is generated via JS, but the HTML Parser will only give you a static view (similar to performing e.g. a curl on an HTML file, or running your web browser with JavaScript disabled). To get access to mentioned content you can have a look at the Selenium Nodes which address exactly this kind of webpages and web app: S…

html parser. Some elements missing.

Hello, I’m tring to get the results of a table in an html [html parsed_vs_original] web page. I used the palladian html retriever and parser. I have the Problemthat some elements in the parsed html source are missing. in the picture you can see the difference between the parsed html in KNime and the original one from google Chrome. Do you know if there is a solution or an ylternative way to get this table ? Thank you, Roberto here the workflow too.KNIME_html.knwf (12.0 KB)

But do note that Skyscanner seems to have an API, so this should probably be your first address.