Quite a few KNIME users especially in industry have been asking about the quality and maintainance of the Community Contributions. Most community projects have very high quality, are well tested and the developers are highly responsive, however we currently don't have any means of checking and ensuring this over longer periods of time. Therefore we had a discussion with all present community developers and some users at the KNIME UGM last week.
Trusted Community Contributions
The idea is to create a set of Trusted Community Contributions for which high code quality is ensured by extensive tests. This not only guarantees that current version are "stable" but also that compatibility with new KNIME versions can easily be verified even if the original developers are not working on the project any more. In addition we can easily provide versions that also work with previous version of KNIME. Another criterion will be in accordance with the general KNIME coding guidelines (e.g. look&feel, node behaviour, etc.). If a community projects meets these criteria it will become trusted and appear on the standard KNIME update site.
Developer support
In order to support the community projects in becoming trusted, we will set up a new build infrastructure in the coming weeks, which offers Git in addition to SVN as source code repository, full access to Jenkins, code quality tracking with Sonar, and a KNIME workflow server for maintaining test workflows. During the discussion last week it also became apparent that the current test workflow support in KNIME is rather limited and needs to be enhanced. We need to cover test workflows which also do extensive tests of the used algorithms as well as more simple compatibility checks for libraries that are already well tested by other means.
The trusted community contributions are an open process and nothing is carved in stone. So feel free to leave your comments and/or requests.
Thanks for sharing. Do you already have some projects meeting these criteria? (So the rest of us can have a better idea what this means in practice.)
The git option is very welcome (I am always unsure whether subversive or subclipse will work better).
Do you have an estimate when the improvements for testing will be available?
Is it a strong requirement to work with previous KNIME versions? (Or Java versions? Java 6 seems to be EOL'd recently, even if Mac users do not have Java 7 for all versions.)
It might be a good idea to have a requirement/recommendation that all third party dependencies should be OSGi dependencies. (Eg. not within the node bundle embedded.) In this case I guess there would be nice to have a repository like eclipse orbit available (possibly with the sources if available). This would reduce the possible interactions between the projects, although that would also mean the required versions might disagree. Do you have a best practice?
I guess this is already a requirement to tag the released/stable versions. A detailed (and a summarized) page would be nice to describe the process once it is decided.
Cheers, gabor
PS: Regarding the coding style... Is it also a requirement to use this kind of Hungarian notation? (I am not a huge fan of it, even though I respect Charles Simonyi, but also think that with the font styles in the editor is enough help.)
Do you plan to add testing functionality for views? That would be very welcome (but maybe only my views are too complicated). Is there a coding guideline regarding the views?
The majority of all extensions are currently candidates for becoming trusted in my opinion. As I said it's a matter of comprehensive tests and KNIME standard compliance. Good examples would be the RDKit or CDK nodes because they already have extensive tests and are very actively supported.
Testing
Concerning the testing environment we currently only have a node that checks for strict equality between a reference table and the current output created by the node under test. This poses problems for e.g. random algorithms or in cases where external output (e.g. from web pages) is processed. It depends on your needs and wishes which nodes should be added.
Currently, we don't have any good approaches to test GUI behaviour (i.e. dialog, views, etc.). We do open and close all dialogs and views during testing and check if this works without problems but we don't check the contents. We still haven't found a nice framework for GUI testing.
Compatibility
I would expect that trusted extensions are compatible with the current and the previous KNIME version. This does not necessarily mean that the previous version gets all new features but important bugs should be fixed and that you can install them (which is almost automatically the case due to the different update sites).
OSGi
The external library problem is a re-occuring one. My suggestion is to use existing plug-ins if possible (a good source is Eclipse Orbit). Alternatively you can create a plug-in containing the external library if it may be of interest to others. If it is a exotic library that very likely only you will use, then I believe it's safe to put it into the node plug-in.
Noding guidelines
We don't have any requirements on code formatting. But we have the KNIME noding guidelines which define nodes' behaviour, rules for datatypes, etc.
Thanks for linking the guideline, maybe the solution described there for dialogs would solve my dialog problem, I'll check it. (I think it could be also linked on the Instructions for Developers page or have it as some pages on the tech knime site (do the Developer Guide links it somewhere?).)
Not really related to the trustedness of contributions, but I think the (web-)documentation is also important. Please forgive me to give a concrete example (I hope they do not mind). The Erlwood cheminfo nodes are really nice, but I guess no one would find the 2D/3D Scatterplot gem from the general description (11. Various Viewers (JMol, Vida, Activity Cliffs, Similarity).) if they are not interested in the chem nodes. That node is wonderful, but how would I know it is in the Open Source or in the GPL version of it? (It is in the OS.) How should I know it contains a node like that. I am not sure how could this be measured, but it would be nice if the trusted would be also well documented on the web too (apologies if I have not found the proper webpage of the Erlwood nodes; their node descriptions are well written). Maybe it is not so important if you plan to provide a public search service which can search within the node descriptions of the trusted, labs and KNIME nodes (without installing them) and provide the desrciption and the feature id which contains it.
Your Compatibility section seems to suggest Java 7 features should not be used, backports should be used if we need any of that functionality. Did I understand correctly?
In tests you mentioned the opening/closing of dialogs, views, .... Does it also include testing the preferences with Apply or OK? (Practically: is it a requirement to have valid defaults?)
Do you have a suggested tagging scheme for the version control?
Hi all, I do have the following “small” problem:
I can’t login in to https://community.knime.org/jenkins and also not from Eclipse cause I receive the message that my user/pw is invalid.
I tried my forum user, knime user and the message.
And although I was able to login some months ago.
Can you help me?
Thanks, Dieter