I’m starting to use Selenium nodes for scraping and automatizing tasks in Knime. So far so good until I faced one challenge.
I need to input a date in a form (the current date in fact, because the default month is always January but I need to scrap the data of the current day) but I cannot write the date in the form directly.
Instead, when clicking in the form, there is a datepicker thanks to which I should be able to select the date. I’m struggling to set the date in this datepicker with the FindElements and Click Nodes (I may be setting them the wrong way…). Does anyone have experience with interacting with this type of widget?
Thanks in advance!
I am not using the selenium nodes (but selenium through java) so it might be slightly different.
Just wondering with “cannot input the date directly in the form” - so if you do it manually you cannot type the date yourself in the mask?
In the cases where I have to select a date through a date picker normally I would send the send-keys directly to the input form or you set-value for the element instead of clicking through the mask.
Does the input field not have a value attribute?
I usually try to work around clicking in these HTML/JS date pickers, as this is extremely tedious. Often, there’s a plain
<input> element behind them (sometimes hidden), in which you can simply type the date (using a Send Keys node). For that you’d probably need to make it visible and/or remove the
readonly flag. As this is highly individual: Can you share the page with the input?
Hi AnotherFraudUser and Philipp,
Thanks a lot for your great help and guidance. Using the Send Keys node to send the date to the form worked really really well.
I was previously not able to type the date directly in the form on the webpage (it was displaying weird things/date…) but using the Send Keys node helped to somehow “force” it to the desired date.
Also, my Find Element node managed to find the input and it returned an “id” number that changes everytime I run the WF. So if I run the WF again, it won’t find the input form and Find Element Node fails.
I’m trying a couple of things on my end. But if any of you has an idea, that would be great. Philipp, I would be happy to share the webpage itself with you but unfortunately the site/portal is login/password protected and I’m not the admin…Anyway, I’m attaching the source if that could help.
source.txt (114.9 KB)
great that you manage to input the date!
if the id changes with every run of the Workflow, you could try to find the element by another search function.
(not sure if the Knime nodes have them)
But normaly you can search by xpath/class (which you can just generate through firefox/chrome if you do not know how to use it) or you could search by input field Name (which hopefully will not change)
Instead of the CSS selector, you can alternatively build XPath queries (when you’re using the “select in browser” functionality). To switch to XPath, go to the Preferences → KNIME → Selenium and switch the “Selector construction strategy” (I admit, this is hidden too much and should be in the dialog instead – just made a TODO note):
Probably, the generated XPath expression is already more “robust”. If not, you’d need to hand tailor a CSS or XPath query. Let us know when you need assistance!
 Just checked the posted sample file – the following XPath should do in your case:
Thank you so much AnotherFraudUser and Philipp for the valuable feedback and for the suggestions, I’ll try the XPath queries instead of CSS Selector and I’ll keep you posted. Have a good week-end!
Hi Philipp and AnotherFraudUser,
Hope you’re doing well.
Quick update: just followed your instructions and moved to XPath. Tried //input[@type=‘datetime’] in the Find Element node. It worked! Alternatively, I also tried input[type=‘datetime’] in CSS and it also worked well. The Send Key node helped me to input the current date. Everything is fine then and I can move forward building the rest of the scraper.
Thanks again for your quick help!
Glad to hear! Keep us posted if you have further questions.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.