Spark denomalizer node

Hi, I have used the Spark Normalizer node and after that I was searching for the Denormalizer node but I couldn’t find it. Is it missing or is there another node that can perform this transformation via Spark?

Cause I supposed I could also use the Spark to Table node and then apply the Denormalizer node, but I want to know if there is a way to perform the denormalization also within spark.

Thanks in advance.

Hi @gujodm
we currently do not have a Denormalizer node for Spark. And apart from doing it in a Spark Java Snippet node there is no option to do this right now within Spark. We can only denormalize in KNIME. I will open a feature request for a Spark Denormalizer node.

2 Likes

Thanks @mareike.hoeger,
I hope to see soon the spark denormalizer node…
meantime, can you show me a quick example for perform the denormalization within the Spark java snippet node?

Hi, I also need Spark denormalizer node, could you please let me know when it will be available or kindly share the java snippet code to do it??.
Thanks in advance!!
Eloy

1 Like

Hey @eloyga, @gujodm,
sorry for the late reply. I do not know when we are able to provide a denormalizer node.

Regarding the snippet:
What we are doing during normalization is to calculate the scale and the translation and then calculate the normalization as
scale * value + translation
So what you would have to do during denormalization is to calculate
(normalizedValue -translation)/scale
You get the scale and translation from the PMML Model from the normalizer node.
You will find to LinearNorm value. The norm for 0.0 = translation and for 1.0 = scale + translation.
With those values you should be able to calculate the denormalization. I have created a very basic example workflow to give you an idea.

SparkDenormalizeSnippet.knwf (18.3 KB)

best regards Mareike

1 Like

You can use Spark ML normalizer, within Spark code snippet.

https://spark.apache.org/docs/latest/ml-features.html#normalizer

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.