@Daniel_Weikert this is what I have on Multi-class problems. Comparing the LogLoss. There was another question by @rogerius1st concerning ML frameworks and KNIME in general which I have tried to answer here:
TL;DR: AUC/Gini for general evaluation (combined with Top Decile Lift), AUCPR/LogLoss for highly imbalanced tasks, RMSE for regression tasks and LogLoss again for multi-class problems (here my experience is limited and you have to be careful how to handle them - you might want to check other measures).
Though obviously there is much more to the quality of a (binary) model (especially the actual cut-off point you choose). There are several graphics and statistics created to help you make the decision - which will have to correspond to your business problem/decision. KNIME has since condensed a lot of that into the: Binary Classification Inspector – KNIME Community Hub. These measures can be used regardless of where your model has been computed. The components would just use a submission=your score and a solution=the known truth and some path variables and R under the hood.
These validation component(s) mentioned in the second part in the link above are described in more details here for
Classification
Regression