Hi guys,
i m using association rule learner for a not-really-large dataset (around 3-4mb). it goes fine till it comes to 98% (saying "executing - item (some changing numbers): level 3. then it takes soo much time and knime nearly crashes. i even tried to run it overnight - it didn t finish. i'm using the same syntax like in a testrun (which worked).
here my configurations:
column containg transactions: bitvector
minimum support: 0,45 (i know this slows down the algorithm, but i need it that low)
underlaying data structure: TIDList (i have far more items than transactions)
itemset type: closed
maximal itemset length: 2
output association rules: tick
minimum confidence: 0,8
i increased the two two memory configurations in knime.ini to 2000mb and 6000mb
any ideas what i m doing wrong?
EDIT:
i gave it a run with min support 0,99 and it worked just fine. so the problem lies within the low support.
but as i know, that the rules i m looking for can occur in 50% of the transactions at most, i want to keep it low.
or is there maybe another possibility what to for this problem?
thanks
I guess the problem is the support threshold which leads the algorithm to generate a huge lattice given your low support value. You might want to decrease the threshold or the memory option (-Xmx) within the knime.ini file.