value selection configuration not reading filtered data

[ciarroccacm]

Hello! I’m trying to create an interactive dashboard using data from different states. I currently have a value selection configuration to allow the user to select a specific state to see certain data from. Upstream from the configuration I have a series of row filters set to exclude rows that are missing data. As a result, a group of states is missing from the data set that is being fed into my value selection configuration. However, the missing states are still showing up as options in my configuration. It seems like the configuration is reading the original data set instead of the filtered data that is directly being linked to it. How do I make the value selection configuration only see the rows from my filtered data (that is being fed into it)?

Also, is there a way to make the drop-down selection menu appear in the interactive view instead of having it only appear in the component configuration menu?

Thanks!

[iCFO]

I think we are going to have to see the workflow to troubleshoot the issue with filtered states still appearing as selectable options in a dropdown.

Widget nodes are what you use to add user interaction controls within an interactive view. Is that what you are attempting currently?

[ciarroccacm]

I substituted my value selection configuration for the widget option, thank you! Here is the updated workflow:

This is the filtered data after going through the row and column filters. This is the data that the widget should be seeing:

These are the values that show up in the selection drop-down. American Samoa and California are two rows that have been filtered out but are still showing up on the drop-down:

[iCFO]

It looks like you are feeding the widget connector prior to the filter. Try running from the filter into the widget connector.

[ciarroccacm]

Which filter do you think the widget needs to be connected to? It is already running from three row filters and then a column filter into the widget. The row filter that the widget is attached to is filtering based on the widget’s flow variable so I don’t think it would be able to be connected there. The filter the widget is currently connected to shows the correct data but the widget isn’t seeing that data.

[iCFO]

I have been trying to troubleshoot from my cell, so I haven’t been able to open the actual workflow yet. What I was trying to say is that it could be an order of flow problem (which can often be quickly fixed by just running an unused flow variable connection to the problem node), but I don’t think that is the issue after another look at the workflow image.

I will open it tonight and trouble shoot in KNIME if it isn’t solved before then.

[iCFO]

It looks like the workflow isn’t uploaded. If you upload a workflow with enough generic data that can illustrate the problem then it will be much easier to troubleshoot. In the mean time here are some quick basics I might try to see if it is a simple refresh issue.

Select all of the nodes then hit F8 to Reset all of the nodes then Execute All and retest the widget.

If that doesn’t work then you can save / close KNIME / restart and then try the above step.

You can try running a flow variable from the filter to the widget. Then Reset and Re-execute

Set the Widget Node for Re-execution on the second tab, an then adjust it a few times on the interactive view.

Delete the existing Widget Node and replace it with a fresh one.

[ciarroccacm]

Thank you! This is my first week using KNIME so I feel like I may have just missed a setting somewhere. I have tried resetting all of the nodes and deleting/replacing the widget with no luck. I had previously expanded all of the components to take the screen capture but put everything back into components for the upload.
all_states_workflow.knwf (180.0 KB)

[iCFO]

all_states_workflow.knwf (55.9 KB)

I was able to work around the issue for you, however I am not able to explain why your initial approach was displaying those filtered out values in the first place…

Also @ciarroccacm that is some solid KNIME work for week 1! Well done. It was not just a simple Widget Node setting issue…

@bruno29a Do you know what caused this unexpected behavior on her initial approach with the Value Selection Widget?

[bruno29a]

Hi @iCFO , I think it’s a bug. I’m able to reproduce it.

I did 2 tests:

In the top one, I hardcoded the same sample data:

Did a Row Filter to remove “American Samoa” and “California”:

We can see they are gone and also the table confirms there are 2 less records:

But I still see the 2 records in the drop down:

In the bottom one, I manually removed “American Samoa” and “California”, which simulates as if the original data comes without these 2 records:

And of course, as expected, these 2 do not show in the drop down:

It looks to be a bug where it’s not using the values from the input port, but rather data either from more upstream or even up to the original table.

Tagging @ScottF @Iris to see if the Knime team can reproduce this on their end and that it’s indeed a bug.

EDIT: I did a 3rd test. I modified my 2nd case to add back the 2 records using the Concatenate:

I added a Sorter node, so that I can see them alphabetically ordered in the Drop down:

However, in the drop down, the 2 added rows appear at the bottom:

I thought may be it’s because of the Row ID, so I even reset the Row IDs with the RowID node, still same result.

So, it seems like the widget is not reading the data coming from the input port, but rather data from upper stream than the immediate node just before it.

[ScottF]

I think this is a situation where the deep magic of the Domain Calculator node can help us.

I used a Domain Calculator in @ciarroccacm 's original workflow between the two components to reset the domain for the Province_States variable, and everything seems to work OK after that.

@bruno29a, I don’t have your WF to check - can you test that this solves the problem in your example as well?

(I know that this is an unintuitive solution for the problem, to say the least. Making domains more accessible and understandable for users is something we’re discussing internally.)

[iCFO]

Hey @ScottF,

That node seems to solve the issue on my tests. I had assumed that the “domain” nodes were all centered around web domain manipulation / management. Are there currently any existing materials out there that help explain how domains are used within KNIME?

[ciarroccacm]

Thank you @iCFO and @bruno29a for playing around with my workflow to make it work.

@ScottF I added the Domain Calculator in my workflow and it fixed the issue.

Thank you all so much for your help!

[ScottF]

Unfortunately we don’t have explanatory material about domains readily available. I’ve posted about this concept a bit before on other threads, but we don’t have detailed resources on it as part of our documentation (yet).

Hel with the node Domain Calculator

Tables in KNIME have what’s called a “table specification”. You can see this by going to the Spec tab in a tabular output port, like this: [2020-07-22 10_45_07-Random data with cluster ID - 0_38 - Data Generator] You can see at a glance that I have 248 rows and 5 columns. Also listed are the ranges of each numeric variable, and the possible values shown for string variable (Cluster_0 through Cluster_3). Based on what we see here the number of possible values for the Cluster Memberships is 4. …

[iCFO]

Thanks @ScottF

Following back through the forum reference links helped me to understand the issue a bit. I still don’t understand why we wouldn’t just want to have all nodes designed to refresh domains and avoid a ton of issues altogether… Is there a large performance hit to refreshing domains for each node, or some other benefit to nodes reading original data / upstream domain info?