Hey Phil,
during a long ride I had some time to play with the workflow.
Generally: Really a cool idea scraping these search suggestions, approximately 1,238,236 ideas what to do with that came to my mind
Some specific recommendations regarding the workflow:
-
I noticed that when executing the workflow, sometimes the suggestions would show, sometimes not. Likely it’s due to the fact, that automaton “types” too fast, and thus the suggestions are not always triggered. There seems to be an easy fix for that though: Pushing the cursor down button explicitly triggers the menu. You can simply achieve this by adding another Send Keys node after the one which enters the query and which sends the key combination
\uE015
(this code denotes thebutton – see the node docs)
-
You had a Wait node to ensure that the suggestions are opened before taking the next step. “Dumb” waits have the disadvantage that you either wait for too long (b/c the results are there earlier), or not long enough (slow Wifi network, hello Deutsche Bahn). You can make this smarter by remove the Wait node and setting a conditional wait in the following node:
This will resume execution immediately, no matter if the suggestions appear
or
style.
-
I changed the CSS selector in the Extract Text to grab the suggestions from the auto-generated one to just
ul[role='listbox']
which might be a bit more robust if the page structure should ever change. -
I definitely like the idea of using the HTML Parser just to extract the results “statically”.
Here’s my modified workflow:
Did you have any success with the counterpart for Google, @kowisoft ?
–Philipp