Read table from website

[Max98]

Hello,

I am new to Knime and I ran into an issue. I try to extract a table from this website: Audi Europe Sales Figures

Unfortunately the whole table is put into one single cell. I want to scrape other tables too ( number of rows can deviate).

I would appreciate your ideas! Thank you in advance!

[Max98]

I tried something like this…

[qqilihq]

Hi Max98,

I Suggest to have a look here about how to extract tables with the Selenium Nodes:

Finance data retrieval..

Hi, I am trying to extract this data table [Invest_Table] from the website: COTY Earnings Date & Report (Coty Inc) - Investing.com. I have encountered couple of issues: The retrieved webpage is larger than 2000 rows (can be retrieved via GET or HTML retriever) and the data table is also after 2K rows thus the proper XPath was difficult but with the help of another tool, it was resolved Couldn’t find an option to concatenate a string at the end of another string in String manipulation node, a…

–Philipp

[Max98]

Thanks a lot! I changed my approach now.
At the moment I am reading 2 titles and 2 tables from this website.

How can I loop this workflow, so that I can read the other pages (same structure) from my excel reader?

Kind regards,
Max

[qqilihq]

Hi Max,

looks good!

For accessing multiple pages, the idea with the loop is right. You would typically connect the red flow-variable port to a Navigate node (which follows the Start WebDriver) and which sequentially navigates to all the pages you want to process.

Configure the “Navigate" node so that the URL is taken from the flow variable.

Does this help?

Best,
Philipp

[Max98]

Thank you very much! One (hopefully) last question. Where can I put the loop end? If I put it directly behind the navigate node, it only lists the URLs (as it should).

I want to create one excel file per loop, but the excel writer has no output for the loop end. So where do I end my loop?

Maybe a stupid question…

Kind regards,
Max

[qqilihq]

Hey Max,

@Max98

Where can I put the loop end? If I put it directly behind the navigate node, it only lists the URLs (as it should).

I want to create one excel file per loop, but the excel writer has no output for the loop end. So where do I end my loop?

Then you’ll need to put it after the Excel writer. As you notice, this one doesn’t have a port, so here’s what to do:

1. Add a Variable Loop End node to the workflow

2. Drag a connection from the upper right corner of the Excel Writer node (there’s an invisible port at that point) to the red circle input of the loop end node.

3. If previous step sounds like black magic, you can right click the Excel Writer and select the menu item “Show flow variable ports”. This is the same as before, but makes things more obvious:

   

Hope this helps!

–Philipp

[Max98]

Thanks a lot! Now it works! My only problem, which still occurs is, that the first table on page 2 has the table id=“table_1” instead of “table_2”. Therefore my Find Elements node doesn’t work anymore… Still something to figure out…

Kind regards, Max