Increase date by looping

Hi,

I want to get data on a website. I stuck when I want to gate the data on a date range. Say the range are from January, 1 2019 till now.

here is the url on a website
image

How can I make the date increase so I can get all data on a website in specific range date. I think it should be use one of many loop node but I don’t know which one that I have to used.

Thank a lot,
Nanda

Hi @Nanda_Rukmana,

I think you do not need loops at all.

You can use the Create Date&Time Range node to have the date range and then use the String Manipulation node with this expression to create the URLs:

join("https://news.detik.com/indeks/all/1?date=", regexReplace($Date&Time$, "(\\d+)-(\\d+)-(\\d+)", "$2/$3/$1"))

Having the URLs you can read them all at once using the HTTP Retriever node and the HTML Parser node. After that you can use the Xpath node to extract data you need.

Here is the workflow to do so: (I have extracted titles)
date_range.knwf (12.9 KB)

:blush:

P.S. This blog post may be helpful as well:

6 Likes

:white_check_mark::medal_sports:

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.