Validate Workflow - Data Preparation

Hi Knime experts,
i am interested in knowing how we can work in an environment where:

  • End users have access to a certain number of data sets
  • End users can create workflows to read and prepare data from these data sets (i.e join, filter, grouping…)
  • End users can SUBMIT the workflows to a central team for validation to control / manage the different data activities (i.e prevent duplicates, identify inefficiencies, detect errors, share with other parts of the organization…)
  • Central team can “approve” / “validate” the workflow before it goes to actual usage in production

Can this be managed through Knime?

thanks,

Hi,
I think you should check out KNIME Server and also ask the KNIME Teammembers (@ScottF @ipazin ) here for more guidance. If all your requirements are possible then this is probably the way to go.
br

4 Likes

Hello @melduque,

and welcome to KNIME Community!

From what you wrote seems you are more interested into process than actual data science/input&output/manipulation/transformation/visualization possibilities around KNIME. Think that latter (along with usability) is something you should evaluate first and if that suits you and your team/company you can start thinking how to build environment around KNIME to support your process. From a quick look the above mentioned requirements should be supported with KNIME Server. I say should cause I’m not a Server user.

For more around KNIME Server see here:

@Daniel_Weikert I’m not a KNIME Team Member, just a support Forum guy :slight_smile:

Br,
Ivan

4 Likes

Sorry Ivan, :sweat_smile:
based on your contributions here (highly appreciated by the way) I thought you have to be
br

1 Like

@melduque This sounds like a lot of jobs the KNIME server can do or help with. I think it would make sense to contact the KNIME team directly and schedule a talks with them.

KNIME server offers scheduling and cooperation and also the possibility to create Serverausfall user groups with certain rights. Concerning the „approval“ process it might be you would have to use something of a two step process. KNIME server offers versioning and comparison of these versions. You might have a shared space where users can contribute but then a super-user/admin would copy the latest version to the actual production workflow. It would involve some planning to set that up.

Concerning the data. In such scenarios the data often resides in databases like ORACLE or Big Data systems. KNIME has a lot of connectors and the KNIME server can also manage connections (you would have to come up with an access concept - who is allowed to see and write what).

I have set up a collection about KNIME server and the KNIME hub (which also has functions that might be interesting to you):

Also you might want to check out the Gartner reviews of KNIME server users. And you can ask further questions in the forum of course. Also KNIME might be able to connect you to existing KNIME server customers to get an idea what the server can do.

2 Likes

You can do it here @melduque

No problem @Daniel_Weikert and tnx :wink:
Ivan

2 Likes