BMMR2: Breast Multiparametric MRI for prediction of NAC Response Forum

Go back to competition Back to thread list Post in this thread

> Evaluation: AUC and wCV metrics

What is the relation between the competition on the mandatory AUC metric and the competition on the optional wCV metric?
Is it like two separate competitions - one competition on AUC and another competition on wCV?
What happens if there is a tie on AUC between two groups and one group submits also wCV while the other doesn't?

Also, I assume that for each team the best submitted result is countered as the final result, even if that result is not the team's last submission. Is that correct?
What happens if the team's best AUC and the team's best wCV are not in the same submission?

Thanks, Simona

Posted by: simona-rc @ Nov. 17, 2021, 10:43 p.m.

Hi Simona:
The wCV repeatability results will likely not be used for any ranking - they are just a secondary analysis of interest. they might be of interest to the groups as establishing any measurement as. a biomarker likely will require demonstration of repeatability along the QIBA guidelines.

Since there are no prizes if two metrics have the same AUC we would just rank them equivalently - they would be "tied". Actually, given the error bars in any measurement I suspect that the statistical analysis will result in many metrics being considered equally ranked; e.g., AUC of .750 and .751 would not be considered to indicate a real difference between two measures.

All results will be analyzed and ranked independently, so yes the best result for a team would be the top one, regardless of how well or poorly other submissions from that group did.

best regards,

Posted by: Newitt @ Nov. 23, 2021, 8:58 p.m.
Post in this thread