Create an port object composing several tables

Hello community,

On the course of developing my own nodes and features, I was able to create custom port objects storing Java objects, which load and save themselves using ModelContent (RO and WO). Now I encounter a problem to create port objects made of several DataTable: I don’t know what would be the official way to load and store them.

Motivation for this strange idea of storing several data tables as one object: the node manipulates complex graphs made of several sets of entities having different attributes (for instance: households, people, dwellings). These entities might be linked together. So far it seems like the best idea to store that data in KNIME is to have one table for links storing the ids of the linked entities, and one table per entity type.

So back to my problem:

  • I created a MultiGraphPortObject implements PortObject
  • so I can declare a Serializer extends PortObjectSerialize which can load and save objects from PortObjectZip*Stream

to save a datatable t so far, I’m using:
ZipEntry zipEntryName = new ZipEntry(tablename);
out.putNextEntry(zipEntryName);
DataContainer.writeToStream(portObject.t, new NonClosableOutputStream(out), exec);

This works, as it saves the table as a file inside the zipfile. But this leads to a slow and sytematic copy of the entire tables between two nodes. It seems like tables are usually dealt with in another way. Should I just store their BufferedTableId instead?

Tks for your help on this specific & complicated question!

Sam.

Hi @samthiriot,
This is a very interesting challenge, I think you should to use a dedicated graph storage data format to transfer your data between nodes instead of relying on a mix of KNIME tables. They aren’t meant to be used that way, as you have seen with the data slowdown.
In the KNIME Network extension, where we have a dedicated data port for networks, the networks are serialized to an XML string for data transfer.

If you have any further questions, don’t hesitate to ask them here :slight_smile:

best,
Gabriel

Hello samthiriot.

I am using Geospatial Operations Shapefile reader to read the Japan map sheapefile.

However, it doesn’t seem to correspond to Japanese and doesn’t read the character well.

Is there any good way?