Text Mining

[ScottF]

This sounds like the classic rare event problem. You might check the threads I’ve linked below - this is something that comes up now and again:

Optimization Options

Hi, I’m trying to do an analysis on binary, very rare events (0.3% probability of occurrence). I’ve tried logistic and random forest predictors, and they both predict 0 events with 99.7% accuracy. I’m assuming the algorithms are optimizing on accuracy. Are there other options to optimize? The two things I care about are: of true positives (more the merrier) True positive/positive ratio (closer to 1 the better) I don’t really care about accuracy all that much.

Unbalanced data - good practice

Hi Everybody, I have a question concerning treatment of unbalanced data. I am trying to “machine learn” model which should detect corporate credit issues with Random Forest using KNIME. After cleaning the data i have enough records with couple of features but with proportion = 1 issue : 250 not-issue. without any modification i have high (99%) accuracy but all categorized as “not-issue” (no detection of “issue” in validation dataset) - no suprise i suppose i have few feature w…