Problem with reusing existing packages/extensions to develop new node extension

[nmorti]

Hi,
I want to create new node extension called NewDBInsert which adds some functionality to already exsiting DBInsert node.
To be more precise, my goal is creating new implementation of org.knime.database.agent.writer.DBWriter.java instead of org.knime.database.agent.writer.impl.DefaultDBWriter.java. Here are my codes for model and factory classes.
NewDBInsertNodeModel.java:

public class NewDBInsertNodeModel extends DBManipulationNodeModel<InsertParameters> {

    public NewDBInsertNodeModel(final DBManipulationConfig config, final String statusColumnPrefix) {
        super(config, statusColumnPrefix);
    }

    @Override
    protected InsertParameters createParameters(final UIParameters uiParameters, final InputData inputData,
        final String[] setColumns, final String[] whereColumns,
        final DataTypeMappingConfiguration<SQLType> knimeToExternal, final DMLStatusListener statusListener) {
        return statusListener == null // Forced line break.
            ? new InsertParameters(uiParameters, inputData, setColumns, knimeToExternal)
            : new InsertParameters(uiParameters, inputData, setColumns, knimeToExternal, statusListener);
    }

    @Override
    protected DMLResult executeWriter(final ExecutionMonitor exec, final DBSession session, final DBWriter writer,
        final InsertParameters params, final DBTable table) throws Exception {
         // I want to use my own implementation for insert
        return writer.insert(exec, table, params);
    }
}

NewDBInsertNodeFactory.java:

public class NewDBInsertNodeFactory extends NodeFactory<NewDBInsertNodeModel> {

    private static final String TYPE = "Insert";

    private static final boolean CONTAINS_SET_COLUMNS = true;

    private static final boolean CONTAINS_WHERE_COLUMNS = false;

    private static final boolean ENABLE_SET_COLUMNS_ENFORCE_INCLUSION = false;

    private static DBManipulationConfig createConfig() {
        return new DBManipulationConfig(CONTAINS_SET_COLUMNS, CONTAINS_WHERE_COLUMNS,
            ENABLE_SET_COLUMNS_ENFORCE_INCLUSION);
    }

    @Override
    public NewDBInsertNodeModel createNodeModel() {
        return new NewDBInsertNodeModel(createConfig(), TYPE);
    }

    @Override
    public int getNrNodeViews() {
        return 0;
    }

    @Override
    public NodeView<NewDBInsertNodeModel> createNodeView(final int viewIndex, final NewDBInsertNodeModel nodeModel) {
        return null;
    }

    @Override
    public boolean hasDialog() {
        return true;
    }

    @Override
    public NodeDialogPane createNodeDialogPane() {
        return new DBManipulationNodeDialog(createConfig(), TYPE);
    }
}

Project Structure:

The problem:
when I want to test the node I fail with error indicating that NewDBInsert couldn’t be created because of java.lang.ClassNotFoundException: org.knime.database.node.io.DBManipulationNodeModel cannot be found by org.kimia.examples.newdbinsert_1.0.0.qualifier
And this prevents me to go to next step and implement my own writer.

Note: I’ve added org.knime.database.nodes and org.knime.database to my plugin dependencies.

Is there any access policy or similar things to using already existing packages?
Should I consider anything special to add dependencies?

The full error log:
ErrorLog.txt (4.4 KB)

[gab1one]

Hi @nmorti,

The package containing the writer node is not exported, this means it is not accessible to other plugins.
If you want to register a different DBWriter implementation, you can do so via the Agent extension point.

See this example for the postgres db, the db writer registration in the plugin.xml:

   <extension point="org.knime.database.Agent">
...
      <Agent
            dbTypes="postgres"
            factory="org.knime.database.extension.postgres.PostgresAgentFactory"
            interface="org.knime.database.agent.writer.DBWriter"/>
...
</extension>

And then the implementation in the AgentFactory:

public class PostgresAgentFactory extends AbstractDBAgentFactory {

...

    /**
     * Constructs a {@link PostgresAgentFactory}.
     */
    public PostgresAgentFactory() {
      ...
        putCreator(DBWriter.class, parameters -> new DefaultDBWriter(parameters.getSessionReference()));
        ...
}

Here the DefaultDBWriter is used, but you could use the same mechanism to register your custom one.

best,
Gabriel

[nmorti]

Hi @gab1one ,
I understood from your reply that doing any changes is so costly and there is no way to change DefaultDBWriter itself. I should implement something like MyOwnPostgres as a new agent which is 99% same as already existing Postgres
This issue get worst if I want to modify DefaultDBStructureManipulator which is used in various databases(e.g, mysql, mssql, etc).

A good scenario could be like changing those classes then building project and launching the knime. is it possible?

Also it would be very appreciated to clarify me about below questions:

1. What exactly Agent is ? What’s it purpose?
2. How a new agent (e.g, MyOwnPostgres) can be implemented as a new node extension project ? please give me the detail as much as you can. like project structure, classes in src and configurations in plugin.xml

[nmorti]

Dear @gab1one,
As said, I wanted to do some changes in DBInsert->executeWriter in order to add some extra functionality in implementation of DBWriter->insert which many databases can connect to it, but because of the related packages not exported (I can’t realize why) I face with such a huge coding!
According to your mentioned example, postgres uses DefaultDBWriter but what about Mysql (I couldn’t find anywhere that DefaultDBWriter used) ? when Mysql connect to DBInsert node, it uses the insert method of this node, how can this use my own implementation of DBWriter->insert() ?

It doesn’t make sense to me that because of just one node, I change all other nodes that can be connected to

[tobias.koetter]

Hello @nmorti ,
usually we do not export the node model implementations. This would make them part of the public API which would make future changes more complicated.
As Gabriel already mentioned the preferred way here would be to write your own DB type and agent implementation. Your DBAgentFactory can extend an existing agent and thus minimizing the needed code. For an example see the RedshiftAgentFactory which extends the PostgresAgentFactory to overwrite the DBLoader agent. All agents that you don’t register are taken automatically from the DefaultDBAgentFactory so you only need to add your new agent.
Bye
Tobias