I am new to using Knime and I am attempting to develop a user interface that will enable our team to extract data from our delta lake with ease.
The database connection is directly linked to a Databricks environment, which means that any major computations will be performed on a separate cluster rather than within Knime. My goal is to create a streamlined approach for filtering requests to the delta lake based on three user-selected conditions that are available in three dropdown menus, such as the Value Selection Widget.
The selection process is hierarchical, meaning that selecting Parameter1 will subsequently filter Parameters2 and 3. In turn, selecting Parameter2 will filter Parameter3 but not Parameter1, and so on. Ultimately, I would like to implement a “refresh” button that utilizes these parameters to filter the entire delta lake. The filtering of the data lake will be used with pyspark code / Spark row filters where the parameters will be used as Flow variables.
However, I am currently facing a basic problem. I am unsure how to create a component that waits for the user to specify the parameters in a particular order before pressing the “Refresh/Execute/Whatever” button to filter the delta lake. Additionally, I am seeking guidance on where to find helpful resources for this task.
Unfortunately, I do not yet have a clear flowchart for this process to present. However, I hope that my intentions are somewhat clear.
Thank you for your assistance.