Purpose

The diagnosis of multiple sclerosis (MS) requires the assessment of lesion load from brain MRIs. Traditionally, MS lesions are manually annotated by radiologists, a process that is inefficient and error prone. The AI-software mdbrain leverages deep-learning to automatically segment MS lesions. Here, we assess the accuracy of the lesion-segmentation algorithm to be released in mdbrain 4.5 compared to SPM-SLS (http://atc.udg.edu/salem/slsToolbox/) and to the inter-rater performance of 4 experts.

Materials & Methods

mdbrain uses a deep neural network to segment lesions from a FLAIR scan. The network was trained using 280 annotated FLAIRs. Performances were tested on a separate dataset of 30 FLAIRs annotated by 4 experts. To assess segmentation accuracy, we computed the lesion-wise F1 score between each algorithm (mdbrain and SPM-SLS) and rater, averaged across raters. The inter-rater F1 was computed by comparing the annotation of each rater against the remaining 3. F1 scores were also computed for different lesion classes separately.

Results

mdbrain achieved an F1 score of 0.72, which was larger than SPM-SLS (F1=0.55) but slightly smaller than the inter-rater (F1=0.75). F1 scores of mdbrain were larger than the inter-rater for juxtacortical (mdbrain F1=0.75; inter-rater F1=0.72) and infratentorial lesions (mdbrain F1=0.58; inter-rater F1=0.55), but smaller for periventricular (mdbrain F1=0.74; inter-rater F1=0.77) and deep-white matter lesions (mdbrain F1=0.70; inter-rater F1=0.76). An average time of 2 minutes was required by mdbrain to process a single scan (GPU-equipped machine).

Discussion

The AI-software mdbrain 4.5 achieved a lesion-segmentation accuracy comparable to a pool of human experts and considerably higher than SPM-SLS.