Linear correlation: node is unnecessarily pessimistic about "low variance"

Dear Knimers,

Suppose I have a table with two columns filled with some random numbers:

image

…and I want to calculate the correlation matrix:

image

The result is:

image

Multiplying or dividing one of the columns by a constant should not affect the correlation coefficient:

image

And indeed:

image

However, when I increase the denominator of the division from 1000.0 to 10000.0:

I get the warning

and the node refuses to calculate a correlation coefficient:

image

With the standard Double renderer the column indeed looks constant:

image

but with the full precision renderer we can see that it is definitely not constant:

image

However, the renderer has no effect on the outcome.

Multiply the column by 10 and all is well again:

image

This is a bug, right? By the way, I am getting a “deja-vu” feeling here, but I cannot find any similar post on the forum.

Best,
Aswin

Hi @Aswin,

yes, you are right! This is definitely a bug. We will look into it and hopefully fix it soon (internal reference: AP-4571 and AP-4564). Sorry for the inconvenience. I hope you can for now use the workaround and just multiply your values.

Best,
Janina

2 Likes

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.