Competition

Evaluation Metrics:

• AUC for prediction of pCR following NAC
• wCV for repeatability of quantitative DWI measures

Submission Guidelines

Each submission shall consist of a single compressed file consisting of two or three component files:

1)    A comma-delimited (.csv) results file (BMMR2_<metric name>.csv) with two columns:

1. PID                                    6-digit patient ID number
2. <metric name>                 Values of the proposed metric for all patients in phase
                                             (Training or Test cohort)

2)    A descriptive file (*.csv or *.xlsx) (BMMR2_<metric name>_desc.<ext.>) including:

1. the metric name
2. a brief description of the metric derivation, including information on what MRI parameters where factors used in the calculation.
3. yes/no flags indicating which if any of the following derived objects provided with the TCIA collection were utilized in the metric calculations:

                                               i.     DCE maps or FTV segmentations 

                                             ii.     DWI derived TRACE images or ADC maps

                                            iii.     DWI manual whole-tumor segmentations

3)    A .csv test/retest results file (BMMR2_<metric name>_TRT.csv) with three columns:

1. PID                                    6-digit patient ID number
2. <metric name>_TRT0       Values of the proposed metric using the 1^st DWI scans   from the 71 test/retest studies
3. <metric name>_TRT1       Values of the proposed metric using the 2^nd DWI scans from the 71 test/retest studies

The 3^rd file may be omitted if DWI test/retest repeatability is not applicable to the metric. e.g., if the metric uses only DCE parameters.

Please do include this file in submissions even if it is identical across multiple ones, e.g., for a metric submitted for both the early-treatment T1 timepoint and the T2 timepoint.

An example submission is available for download Link to download sample TRAINING and TEST submission.

Submission limits

Training phase:    Research teams may submit any number of submissions to the MedICI challenge hosting site.

Test phase:         Research teams may submit up to 10 individual metrics for consideration in the challenge. No re-submission will be allowed during this phase.

Metrics requirements

1)    Metrics may be of any type but must produce a scalar numeric value for each patient that is to be tested for predictive power for pCR. For example, Mean ADC at T0 and Mean ADC at T1 would be 2 separate metrics.

2)    Values must be provided for all patients in the appropriate cohort (Training or Test) and for both first and second DWI acquisitions in the n=71 test/retest cohort.

3)    Metric calculations may use any combination of the provided T2w, DWI and DCE image acquisitions and the associated clinical data, including tumor subtype.

4)    For definitiveness in the AUC calculations, please adjust the overall sign of all metrics so higher metric values indicate greater predicted likelihood of pCR.

Terms and Conditions

By participating in this challenge, each participant agrees to:

1. Not share the challenge data with anyone not involved with the BMMR2 challenge

Note:

Results of this challenge will be broadly relevant to the use of DWI for breast cancer treatment response assessment. In particular, they should be applicable to improving patient treatment decisions in breast cancer neoadjuvant clinical trials.