Selenium Nodes in Knime

#1

Hello!
I am trying to use Selenium nodes for filling out a webform with a data set.
I got them to work to connect, and submit the form and get the results back for one item.
How do I connect a table or excel file to selenium nodes and how do I export the data into excel from selenium nodes? I am not able to find anything. Please help!

0 Likes

#2

Hi,

If you have all the field values for the form in a dataset and want to fill the form multiple times and get the results back, you need to do that in a loop.
I suggest using Table Row To Variable Loop Start on the dataset and pass the values row by row as flow variables to the main flow where you are using Selenium nodes to fill the form. After filling the form and submitting it, you can take the results and close the loop using the Loop End node.

If you need more detailed solution, I need more detailed explanation. Don’t hesitate to ask more questions if you need to.

:blush:

3 Likes

#3

Did just that and it worked! I have a bit of an issue with the collected data file though. Since I used csv writer and append action, i get my results appended during the loop. However, when i restart the workflow I need a nee file to write results i - imagine I am doing it monthly and need the new file to start the workflow each month. How can I achieve that without changing node configuration settings each month?

0 Likes

#4

Use Create Date&Time Range node to create the current execution date as a single cell then convert it to a flow variable by using a Table Row to Variable node and pass it to the Create File Name node and create your file path for the CSV Writer node.

:blush:

3 Likes

#5

This takes care of the putting the date in the file name, but I don’t believe it takes care creating the new file each run, does it? For now CSV writer node is within the loop of writing web results into the file. Should I create the empty file before the loop and append rows within the loop? If so, how do I create an empty file first?

0 Likes

#6

If you run the workflow once a month, it would create a new file each time since the file name is different each time. If you like to have different files in each loop, you can add the time as well, then it will create separate files in each loop. But I think you want it to be the same file in each workflow execution so when you use date, the file name is the same that day. When you run the workflow some other day, since the date has changed, the file name is different and it creates a new file.

No. You don’t need to. The append method appends the table if the file already exists. If the file doesn’t exist already, it will be created.

:blush:

1 Like

#7

Thats if I set the file name to contain the date and run it on different dates. If I run the file a few times a day and thus have date&time, wouldn’t each appended record have a different date&time of execution and thus create different files?

0 Likes

#8

If you use only “date”, it would be the same file all day long.
If you use “date&time”, you would have different file each time the node is executed since every second you get a new file name.

If you want to have the same file in each run but different files in several execution of the workflow on the same day, then use “date&time” but modify it with string manipulation node before passing it to the create file name node. For example you can keep the date and the hour part of the name so it would be the same file within the hour of the execution time.

:blush:

1 Like

#9

Seems like I got it to work. Will report back.

I have a further question on Selenium nodes. I got the workflow to work via Find Elements and Send Keys nodes. Now, if the data is correct, it works. However, if the Find Elements fails, the workflow fails (example: I submit a registration number and expect the result on another page. However, if I submitted the wrong registration number, the website shows an error and does not present me the expected page). How can I get the workflow to work as follows:
if success = do what is already designed, if failure = record the error message (or at the very least record the word ‘error’) and get to the next record in the loop?

0 Likes

#10

Check the “Return missing values when nothing is found” option in the configuration window of the Find Elements node. That way you get a ? (missing value) when the element is not found and the workflow goes on normally till you don’t call that value (which is now missing) in the next steps.

:blush:

2 Likes

#11

Did that, still get an error.

0 Likes

#12

In this case you can use error handling nodes:

:blush:

1 Like

#13

This is a never-ending issue it looks like… So my loop worked before I used an Error Handling node, and the workflow was breaking for the right reason. Now that I used Catch Errors (Data Ports) I get the following error “Encountered loop-end without corresponding head!”. I am not sure why it even goes to the loop end after the first row in the loop?
I am so close, I need it sorted :slight_smile:

0 Likes

#14

When using the error handling construct in a loop, use a table which has exactly the same table structure as the main flow input for the second input port of the Catch Errors node or check the “Allow changing table specification” option. In your case the former is preferred.

Here are two examples in a workflow:
Selenium_Error_Handling.knwf (131.4 KB)

In the top flow, I used the return missing value option.
In the bottom flow, I used the error handling construct. If you check the Table Creator (Node 14) you can see that I have provided the same table structure as it is in the output of the Column Filter (Node 23).

:blush:

2 Likes

#15

I really appreciate your help, Armin! I guess it is becoming a bit more complex for me to follow. I am attaching my workflow.
Find Elements node next to Node33 is the one that is failing on the Jim Bim record because the business number is incorrect.
What I am failing to do is 1. catch the errors and 2. on error report this business number as error and 3. redirect the flow to do something else (in this case, press on ‘return to welcome page’ and go through the terms and agreement again while using the next in line account). Seems like this task is over my head. I am hoping you can help me achieve this or something close it. Thank you so so much!
.Commodity Tax Validator for Forum.knwf (64.5 KB)

0 Likes

#16

I modified your workflow:
Commodity Tax Validator for Forum.knwf (72.2 KB)

:blush:

1 Like

#17

Thank you for modifying it! It is a ted different though: you send every result through ‘Return to welcome page’. This is not the intent. For the successful result it needs to envoke New Search while for the unsuccessful it has no other choice but to return to welcome page. Can this be achieved?

0 Likes

#18

Use Navigate node and start the loop on this node or use If Switch node.

:blush:

1 Like

#19

I must be dumb. At Missing Value node I added the Row to Variable port, then Rules Engine for variables, then connected to If switch and up to this point I get the correct direction of the port. At this point I am trying to connect Find Elements and the node does not want to connect, it throws an error 2019-06-21_17h08_56|544x208
Why?

0 Likes

#20

Here you are:
Commodity Tax Validator for Forum.knwf (82.5 KB)

:blush:

1 Like