Longitudinal lesion segmentation in Multiple Sclerosis is a very challenging task and often there is not even consensus among expert neuroradiologists whether or not “a white spot” should be counted as a new lesion. 24 teams from 4 continents accepted this challenge and submitted their best algorithms and pipelines. The ground truth was derived from consensus reading of 4 expert neuroradiologists. The performance of the the AI-tools was compared against this ground truth using F1- & DICE-Score as metrics – see example:

In the end mediaire’s approach achieved the best results in terms of lesion detection (F1-Score) and ranked #1 and #3, respectively, while outperforming teams from Siemens, NVIDIA, several well known academic teams and an expert neuroradiologist. However, as neuroradiologists have been involved in generating the ground truth there certainly was a bias in favour of the human rater here…

The full results can be accessed here: MICCAI 2021 New lesion segmentation challenge – Details

The technology used in this challenge is partially available in our commercial product mdbrain already where it is regularly improved based on new data and feedback from our customers.