How to correctly normalize and denormalize validation data for a neural network

[LeoNeo]

Dear KNIME community,
I am currently facing a neural network related problem in KNIME for which I have not been able to find a solution yet. I also searched the forum but unfortunately couldn’t find a solution there either. So in the following I will describe my problem and am looking forward to your help and advice.

I have designed a workflow in KNIME which incorporates a neural network (composed of the RProp MLP Learner node and the MLP Predictor node) that is intended to be used for a regression task. In particular, it is supposed to predict the concentrations of unknown solutions based on pictures (or rather the color intensity) of these solutions. So far, I have managed to get most of it working, which includes the preprocessing of the training pictures, the training of the neural network itself and also the testing of the models performance with a set of test data (as a side note, the pictures of the test set are very similar to the pictures that have been used for the training of the neural network). This is working really fine and the accuracy of the model is great when it comes to predicting the concentrations of the test set.

Now unfortunately, when I try to validate the models performance using a set of pictures that the neural network has never seen before, things get kind of strange and the accuracy gets rather bad. For the validation set, I am using 125 pictures which all correspond to the same particular concentration. The features of these pictures (namely the mean color intensities of the RGB channels) are all very similar since they are just frames taken from a video of one solution. Now the strange part is that when I predict the concentrations of the pictures of the validation set, the predicted concentrations do differ quite a lot even though the features of the pictures are very similar.

My theory is that this has to do with how I am normalizing my validation data set before and after the MLP prediction node. Currently, I normalize the validation set separately from the training and testing set (1) but then denormalize the predicted concentration with a denormalizer node (2) that is connected to the same normalizer node (3), which is used for the normalization of the training and test data. I have the feeling that there is something really wrong about how I normalize and denormalize my validation data in this particular workflow.

Do you have any idea or solution for my problem? Is the normalization done correctly? Any help is highly appreciated!

I added the KNIME workflow in the attachment. Please let me know if you need any further information.

kind regards,
Leonard Engels

[AlexanderFillbrunn]

Hi,
Welcome to the KNIME Forum! Your min and max for the normalization during training should also be used for normalizing the data in your validation data set. Otherwise you will get wrong results. Just imagine that you have values 1, 2, 3, 4 in your training data, but only values 2 and 3 in your validation data. If you normalize them separately, you get 0, 0.33, 0.66, 1 for the data in your training data and 0 and 1 for the data in your validation data. This means 2 has become 1 and 3 has become 4! If you apply the same normalization for both, you will get 0.33 and 0.66 for the values in the validation data and this is how things should be.
One thing you need to be careful about, though: if your validation data contains data that is out of the range of the training data, you will get values smaller than 0 or greater than 1, something that you might want to avoid. In that case you need to apply a normalization that covers the full range, e.g. by using Table Creator + Normalizer and then Normalizer Apply on all your data.
Kind regards,
Alexander

[LeoNeo]

Hi Alexander,
thank you for the quick answer! Now everything works fine! I exchanged the regular normalizer node which I used for the normalization of the validation set with a normalizer (apply) node and connected that to the normalizer node for the training data. Your example also really helped me to understand why it didn’t work before.

Thanks a lot!