Joiner node freezes

Hello,

The joiner got stuck at %99 when joining tables which has 33 million in the left table and 1 million in the right table.

The joiner field has a lot null values. I tried to use run garbage collector and cache nodes. I also tried to exclude null values and join but it still freezes at 99%.

I appreciate your help and time! Thank you!

1 Like

Hi @Erimos and welcome to the Knime Community.

Can you tell us which version of Knime you are using?

Joining on NULLs used to be a problem where it could take hours to run instead of seconds. This was corrected in newer versions.

Alternatively, this is what I used to do when I was using the older version, for Inner joins, I would simply filter out the NULL records before joining, and for Left (or Right) joins, I would row split (separate) the NULL records, join only on the non-NULL records, and then concatenate the NULL records to complete the set.

That is how I optimized the process. These joins would be done in a couple of seconds while originally they were taking up to 8hours to run.

5 Likes

Hi @Erimos , here’s a quick demo to illustrate what I said:
image

Here’s the demo: Join without NULLs.knwf (17.3 KB)

4 Likes

Hello @bruno29a.

I am using version 4.4.0.

I have tried your solution and it worked!

Thank you a lot!

2 Likes