Import a DeepLearning4J model from Java to KNIME

Hi,

I'd like to import a MultiLayerNetwork model build with DeepLearning4J on my Java IDE to KNIME. I tried saving my model on Java and importing it via the node DL4J Model Reader, but it seems like models saved through Java and models saved through KNIME don't have the same structure:

Models saved with KNIME create a dl4j file that once extracted contains files like dnn_type0, isTrained, labelX, layer_typeX, layerX ... and a binary file mln_model.

Models saved with Java create a zip file that once extracted contains only two files: one configuration.json file (very similar to all the layerX files) and one coefficients.bin file.

I tried loading the Java-created file anyway with the DL4J Model Reader, but it gives an empty model.

Do anyone see how to do this ?

Best,

Simon

Hi vdk.simon

in principle it should be possible to do this (however I never tested it myself and I think its not easy). In KNIME we save some additionally information about the model and the data it was learned on compared to saving the model in java. These are the files you mentioned, like dnn_type0, isTrained, labelX etc. which are just text files (just open them with an editor to see the structure). However, the mln_model file should be exactly the same as the one saved with java (you can try to extract it). So we just add some additional text files to the model and zip everything again.

You could try looking at a model saved with KNIME and fake the same structure using your model saved with java. I guess you want to use an already trained model for classification? Then have a look how the models look like after the corresponding training node. This will add information about the column names the model was trained on which need to exactly match if you want to use the model with the predictor node. Also you need to add the values of the class column so the predictor nodes knows what to predict (alternatively you could use the 'Layer Predictor Node' to obtain the activations from the softmax layer and assign the class values manually). Furthermore, it is very important that you use the same version in java as we are using in KNIME which is currently 0.8.0 .

Unfortunately, I see a lot of problems that could possibly arise but it should be doable if you are very careful. Currently, we do not support external model reading but we maybe add that in the future.

If you want, you can provide me with the model you want to use and I can have a look at it if I find the time. For this it would be very helpful to know your use case in detail.

Cheers

David