BMMR2: Breast Multiparametric MRI for prediction of NAC Response

Organized by qin-organizer - Current server time: Jan. 29, 2023, 10:21 p.m. UTC


Open Validation Phase
Jan. 1, 2022, midnight UTC


Open Validation Phase
Jan. 1, 2022, midnight UTC


Competition Ends
Dec. 25, 2021, 5 a.m. UTC

Neoadjuvant chemotherapy (NAC), standard of care for invasive breast cancer, allows non-invasive monitoring of treatment progress with longitudinal MRI scans. Quantitative MRI metrics are needed to accurately predict treatment outcome both for de-escalation of treatment for good responders and for treatment switching for poor responders. The purpose of this challenge will be to compare predictive metrics for pCR derived from MRI diffusion weighted imaging (DWI), alone or in combination with dynamic-contrast-enhanced MRI (DCE).

The image data set for the challenge will be a subset of MRI studies from the ACRIN 6698 Trial ("6698"), a sub-study of the I-SPY 2 TRIAL designed to investigate the use of DWI for breast cancer NAC treatment response monitoring. Data will be available through The Cancer Imaging Archive (TCIA). Subjects in 6698 had longitudinal MRI studies, including standardized DWI and DCE scans, at four time points during NAC: T0 (pre-NAC), T1 (3 weeks NAC), T2 (12 weeks NAC) and T3 (pre-Surgery). T0-T2 will be used in the challenge. In addition to the original DWI and DCE image data, the DWI whole-tumor manual segmentations and DCE functional tumor volume segmentations utilized in the 6698 primary analysis will be provided to challenge participants. Use of the provided segmentations in challenge submissions optional. The patient cohort will be divided into training and test sub-groups, and candidate metrics may be either simple metrics (untrained), or models trained on the training sub-group. Each candidate metric submitted will be evaluated by the statistics group to determine its AUC for prediction of pCR in the test sub-group.

As a secondary aim, repeatability of the DWI measures will be evaluated in a 71 patient sub-group with "coffee-break' style test/retest DWI acquisitions. Within-subject coefficient of variance (wCV) will be evaluated and compared to reference values found for whole-tumor mean ADC from the ACRIN 6698 published analysis.

The BMMR2 challenge is closed to new registrants and for data downloads.

The submission site will remain open until 23:59 UTC on December 21, 2021.

Evaluation Metrics:

  • AUC for prediction of pCR following NAC
  • wCV for repeatability of quantitative DWI measures

Submission Guidelines

Each submission shall consist of a single compressed file consisting of two or three component files:

1)    A comma-delimited (.csv) results file (BMMR2_<metric name>.csv) with two columns:

  1. PID                                    6-digit patient ID number
  2. <metric name>                 Values of the proposed metric for all patients in phase
                                              (Training or Test cohort)

2)    A descriptive file (*.csv or *.xlsx) (BMMR2_<metric name>_desc.<ext.>) including:

  1. the metric name
  2. a brief description of the metric derivation, including information on what MRI parameters where factors used in the calculation.
  3. yes/no flags indicating which if any of the following derived objects provided with the TCIA collection were utilized in the metric calculations:

                                               i.     DCE maps or FTV segmentations 

                                             ii.     DWI derived TRACE images or ADC maps

                                            iii.     DWI manual whole-tumor segmentations

3)    A .csv test/retest results file (BMMR2_<metric name>_TRT.csv) with three columns:

  1. PID                                    6-digit patient ID number
  2. <metric name>_TRT0       Values of the proposed metric using the 1st DWI scans   from the 71 test/retest studies
  3. <metric name>_TRT1       Values of the proposed metric using the 2nd DWI scans from the 71 test/retest studies

The 3rd file may be omitted if DWI test/retest repeatability is not applicable to the metric. e.g., if the metric uses only DCE parameters.

Please do include this file in submissions even if it is identical across multiple ones, e.g., for a metric submitted for both the early-treatment T1 timepoint and the T2 timepoint.

An example submission is available for download Link to download sample TRAINING and TEST submission.

Submission limits

Training phase:    Research teams may submit any number of submissions to the MedICI challenge hosting site.

Test phase:         Research teams may submit up to 10 individual metrics for consideration in the challenge. No re-submission will be allowed during this phase.

Metrics requirements

1)    Metrics may be of any type but must produce a scalar numeric value for each patient that is to be tested for predictive power for pCR. For example, Mean ADC at T0 and Mean ADC at T1 would be 2 separate metrics.

2)    Values must be provided for all patients in the appropriate cohort (Training or Test) and for both first and second DWI acquisitions in the n=71 test/retest cohort.

3)    Metric calculations may use any combination of the provided T2w, DWI and DCE image acquisitions and the associated clinical data, including tumor subtype.

4)    For definitiveness in the AUC calculations, please adjust the overall sign of all metrics so higher metric values indicate greater predicted likelihood of pCR.

Terms and Conditions

By participating in this challenge, each participant agrees to:

  1. Not share the challenge data with anyone not involved with the BMMR2 challenge


Results of this challenge will be broadly relevant to the use of DWI for breast cancer treatment response assessment. In particular, they should be applicable to improving patient treatment decisions in breast cancer neoadjuvant clinical trials.


Start: May 28, 2021, 11 p.m.

Open Validation Phase

Start: Jan. 1, 2022, midnight

Description: Open Validation Phase


Start: July 1, 2021, midnight

Competition Ends

Dec. 25, 2021, 5 a.m.

You must be logged in to participate in competitions.

Sign In