Error in BEDROC measure?

Hi, it seems that the BEDROC meaure calculated by the "Virtual Screening Metrics" node is not correct. The BEDROC values reported by the "Virtual Screening Metrics" node differ from results obtained from a Python implementation "vs_perf.py" by Dimitar Hristozov (available here: http://www2.chemie.uni-erlangen.de/people/Dimitar_Hristozov/sprt_info/) and my own implementation based on the formula introduced in the BEDROC paper by Truchon and Bayly (JCIM, 2007, 47 (2), pp 488–508). Can anyone confirm this? Best regards, René Thomsen, Molegro

Hi!

I 'm starting to use Knime for data classification. When I attemp to configure a classifier Learner, error message appears:"No Column in spec compatibleto nominal value". 

What is the problem?

thanks

Dear Giulia,

I guess you noticed it already but your question is not related to the top of this thread (so please open a new thread for follow-up questions). The error message that you see tells you that there is no column in the input table that is categorical/nominal. Is the node connected to a configured (yellow) file reader? It needs the table meta information in order to be configured.

I have no idea what BEDROC  means so someone else has to comment on René's question.

Regards,

  adfa
  Bernd

Rene,

 

you're right - it has to be wrong because BEDROC is supposed to be bounded between 0 and 1, which it often isn't in the Erl Wood "Virtual Screening Metrics" node.  I'm looking at the source code ATM but don't fully understand it so it'll take time.  I'll try to re-implement this and see what happens.

 

On a related note, the method that calculates the RIE has a mistake.  I'm not gonna post the source code here (it's downloadable anyway), but the final "- 1.0" clause is in a bracket when it shouldn't be, thus is inside an exponential function and thus gives a vastly different answer compared to what it should be.

Ed. 

Relating to the RIE, I have made the maintainers aware outside of this forum, but I can confirm that issue does exist.

This issue has been fixed in the latest (Dec 2014/Jan 2015) release. Thanks for reporting it!

Hey Rene!

I know this is not a solution to your question but would you mind sharing your workflow using VSM?

I’m new to Knime and want to implement VSM but don’t know where to start. And the values that I’m getting are NAN, so want to figure out what am I doing wrong.

Thanks,
Hrshita

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.