Announcement: Selenium Nodes 4 released

Community Extensions Palladian & Selenium
#1

Selenium Nodes 4.0 is available

If you’re installing into a fresh KNIME 4.0, please get the update site URL here.

This release provides no new features, rather it ensures compatibility to KNIME 4.0, and thus involves a breaking change. Due to changes in KNIME 4.0 we had to remove the jBrowser plugin – we feel that this is not a big thing, as (1) it was always an optional installation and thus not widely used anyways, and (2) there are now other (better) options for running a headless, bundled browser (e.g. Chrome, Chromium, or Firefox).

Beside that, we removed the “Execute Selenium script” node – the related “Selenium Builder” project for creating the JSON scripts is not maintained since a long while, and the node caused confusions, as there are other, incompatible Selenium JSON script formats which could not be processed by this node anyways.

For more information, please check the change log.

Let us know if you have feedback, comments, or issues!

– Philipp

5 Likes
Palladian nodes gone after update to knime 4.0
#2

In case you’re updating an existing KNIME installation where the jBrowserDriver has been installed, you need to uninstall jBrowserDriver before, else wise you’ll see a “Cannot complete the install because of a conflicting dependency”.

For that, go to About KNIME → Installation Details → Installed Software and there uninstall “Selenium Nodes for KNIME: jBrowserDriver” by selecting the entry and clicking “Uninstall…”

Afterwards, please run the update again.

1 Like
KNIME 4.0
Knime Update Next button is not moving ?
Knime 4.0
#3

:robot: What You See Is What You Select

Selenium Nodes 4.1 is now available as a free update for all current users. Here’s a quick overview with the new highlights:

mov
mov796×606 1.83 MB

Visual Element Selection

The Find Elements node now has a Select button which allows you to grab page elements more easily than before. Say “goodbye” to writing queries by hand or copying selectors from your browser. Instead, simply point at the desired element in the browser window, click on it , and a matching CSS selector will be created for you. This feature works in all Chrome-based browsers (Chrome, Chromium and Opera).

mov
mov772×600 745 KB

Live Selector Preview

The Find Elements node will show you all matched elements for a given query in realtime — both in the browser window and in the DOM outline. Modify your query and you will immediately see whether you hit the desired elements. This feature is available for all Chrome-based browsers — as stated above.

What Else?

Chris Terrell from EverydayVBA made a three minute video which showcases the new functionality. Watch here and also check out his other videos!

There are several more changes, e.g. a new Reset on Load node. Please check our changelog for all the details.

3 Likes
#4

And there’s a 4.1.1 fix now which resolves an issue when using the new selection tools with the “Start Pooled WebDriver” node.

Happy weekend.

#5

Update 4.2 adds a new predefined snippet to disable SSL-related errors (see here). Beside that, we’ve updated the bundled Chromium browser to version 80.0.3987.117.

#6

We released Selenium Nodes 4.3.1 a few days ago. Notable changes include:

  • A brand-new Create PDF node which allows you to write PDF files using Chromium-based headless browsers. How does it work for you? We’re looking forward to your feedback!
  • Start WebDriver: Input port for the factory is now optional – you can configure the WebDriver directly within the node configuration, the WebDriver Factory is no longer needed.
  • Several new Capability snippets for the Firefox browser (kudos @armingrudd)
  • Updated driver binaries and bundled Chromium
  • Reset on Load: Fix when used in metanode or wrapped metanode or component or however these things are called now (thanks to @Rich_ard)

– Philipp

4 Likes
#7

What snippets were added?

#8

Hi Philipp

The pdf creator works very well. Features It is great to scan a web page with all its links and texts (not in the form of an image or image format) and save it as pdf.

also the web driver features have been improved operationally :+1: :knime: :+1:

3 Likes
#9

Please check the change log for details:

#10

Since a few minutes, there’s Selenium Nodes 4.4 on the update site. The changes:

  • Navigate: Add “Flow Variable” to URL input.
  • Send Keys: Add “Flow Variable” to text input.
  • Wait: Add “Flow Variable” to time input.
  • Find Elements: Add “Flow Variable” to query and result column input.
  • Take Full Page Screenshot: New node for taking “full page” screenhots (i.e. longer than the visible window bounds) using Chrome. Currently labeled as “beta” — your feedback is welcome!
  • Update driver binaries (ChromeDriver 81.0.4044.138, OperaChromiumDriver 81.0.4044.113)

Check the change log for the full chronology.

3 Likes
#11

Congratulations Philipp; There have been much more powerful features with the latest updates.
:trophy: :knime:

2 Likes
#12

Selenium Nodes 4.5 is available. The changes since 4.4:

version-4.5 (2020-08-02)

  • Table Extractor: Allow to skip empty rows (kudos @armingrudd)
  • Update driver binaries (OperaChromiumDriver 84.0.4147.89)
  • Table Extractor: Fixed handling of empty <tr></tr> rows (kudos @armingrudd)

version-4.4.3 (2020-07-25)

  • Update driver binaries (ChromeDriver 84.0.4147.30)
  • Properly propagate proxy settings for Chrome (thanks to @dasturiol)
  • Typing for proxy port settings in Firefox template must be Long instead of String (thanks to @dasturiol)

version-4.4.2 (2020-07-15)

  • Update driver binaries (ChromeDriver 83.0.4103.39, OperaChromiumDriver 83.0.4103.97)

version-4.4.1 (2020-05-20)

  • Navigate: Prevent validation errors when using Flow Variable input
  • Send Keys: Prevent validation errors when using Flow Variable input
3 Likes
#13

Selenium Nodes 4.6 is out. Changes since 4.5 (also see here):

  • (Info) Requires at least KNIME 4.0 (please make sure you’re using an update site URL corresponding to your KNIME version)
  • (Add) Table Extractor: Allow to treat rowspan and colspan as missing cells
  • (Add) Select: Add “Remove input column” setting
  • (Add) Send File: New node for sending a file path e.g. to an <input> element
  • (Add) Extract Attribute: Allow to specify name of output column
  • (Add) Extract CSS Property: Allow to specify name of output column
  • (Add) Extract Outer/InnerHTML: Allow to specify name of output column
  • (Change) Update driver binaries (ChromeDriver 85.0.4183.87, OperaChromiumDriver 85.0.4183.102, geckodriver 0.27.0)
  • (Change) Update bundled Chromium to 85.0.4183.121
  • (Change) Extract InnerHTML: Rename to Extract Outer/InnerHTML
  • (Change) Improve error message when Selenium data is missing; explain to re-run from the “Start WebDriver” node
  • (Change) Extract Attribute: Suggest common attribute names
  • (Change) Hide driver binary settings in preferences
  • (Fix) Select: Saving of settings

Update site URLs (see here)

  • (KNIME 4.2) https://download.seleniumnodes.com/4.2
  • (KNIME 4.1) https://download.seleniumnodes.com/4.1
  • (KNIME 4.0) https://download.seleniumnodes.com/4.0
2 Likes
#14

Hi all!

:partying_face: Here’s Selenium Nodes 4.7 (Codename “Alaaf”)

And there’s one thing which will make it much easier for you to build Selenium Workflows: Many nodes now have an integrated “Find Elements” functionality, which will make your workflows much cleaner, easier to operate and faster to execute. Small change, big impact!

image
When opening existing workflows, you’ll see a notification that some nodes have been updated. Don’t hesitate to click the “Migrate Workflow …” button to update to the latest node versions (a backup copy will be automatically retained and the nodes are of course backwards-compatible).

Changelog

Detailed changelog (also see here):

  • (Info) Requires at least KNIME 4.1 (please make sure you’re using an update site URL corresponding to your KNIME version)
  • (Add) Extract Attribute, Clear, Click, Extract CSS Property, Drag and Drop, Extract Inner/OuterHTML, Send File Path, Send Keys, Submit, Table Extractor: New node versions which allow to pick WebElements directly within the dialog without having to put a “Find Elements” input node first (old versions still work and become “deprecated”)
  • (Change) Extract Inner/OuterHTML renamed to Extract Text and add “textContent” option
  • (Change) Use NodePit Licensing plugin for license key management
  • (Change) Execute JavaScript: Support boolean and long flow variables which finally made it to KNIME 4.1
  • (Change) Execute JavaScript: Support collection flow variables and input cells
  • (Change) Execute JavaScript: Support JSON input and output
  • (Change) Add migration logic to replace deprecated nodes
  • (Change) Update driver binaries (ChromeDriver 88.0.4324.96, OperaChromiumDriver 88.0.4324.104, geckodriver 0.29.0)
  • (Change) Update bundled Chromium to 88.0.4324.153
  • (Fix) Execute JavaScript: Properly map Boolean cells

Update Site

Update site URLs (see here):

  • (KNIME 4.3) https://download.seleniumnodes.com/4.3
  • (KNIME 4.2) https://download.seleniumnodes.com/4.2
  • (KNIME 4.1) https://download.seleniumnodes.com/4.1

Feedback, questions, comments? Get in touch at mail@seleniumnodes.com, @seleniumnodes, or just reply here!

– Philipp

PS: More news – both from Palladian and NodePit – to follow soon!

2 Likes
#15

There was an issue in the 4.7 release which could lead to a conflicting dependency and thus prevented installation on some environments. We just released version 4.7.1 (code name consequently “Aschermittwoch” :stuck_out_tongue: ) which fixes this problem.

Note: It’s important that you also update Palladian to the newest version which we just rolled out, and which is v2.4 (an announcement and overview over the changes will follow). Find the update site URLs here:

Thanks!

Philipp