Error in BEDROC measure?

#1

Hi, it seems that the BEDROC meaure calculated by the "Virtual Screening Metrics" node is not correct. The BEDROC values reported by the "Virtual Screening Metrics" node differ from results obtained from a Python implementation "vs_perf.py" by Dimitar Hristozov (available here: http://www2.chemie.uni-erlangen.de/people/Dimitar_Hristozov/sprt_info/) and my own implementation based on the formula introduced in the BEDROC paper by Truchon and Bayly (JCIM, 2007, 47 (2), pp 488–508). Can anyone confirm this? Best regards, René Thomsen, Molegro

0 Likes

#2

Hi!

I 'm starting to use Knime for data classification. When I attemp to configure a classifier Learner, error message appears:"No Column in spec compatibleto nominal value". 

What is the problem?

thanks

0 Likes

#3

Dear Giulia,

I guess you noticed it already but your question is not related to the top of this thread (so please open a new thread for follow-up questions). The error message that you see tells you that there is no column in the input table that is categorical/nominal. Is the node connected to a configured (yellow) file reader? It needs the table meta information in order to be configured.

I have no idea what BEDROC  means so someone else has to comment on René's question.

Regards,

  adfa
  Bernd

0 Likes

#4

Rene,

 

you're right - it has to be wrong because BEDROC is supposed to be bounded between 0 and 1, which it often isn't in the Erl Wood "Virtual Screening Metrics" node.  I'm looking at the source code ATM but don't fully understand it so it'll take time.  I'll try to re-implement this and see what happens.

 

On a related note, the method that calculates the RIE has a mistake.  I'm not gonna post the source code here (it's downloadable anyway), but the final "- 1.0" clause is in a bracket when it shouldn't be, thus is inside an exponential function and thus gives a vastly different answer compared to what it should be.

Ed. 

0 Likes

#5

Relating to the RIE, I have made the maintainers aware outside of this forum, but I can confirm that issue does exist.

0 Likes

#6

This issue has been fixed in the latest (Dec 2014/Jan 2015) release. Thanks for reporting it!

0 Likes