Comparison of MS segmentation models on MP2RAGE images

[Nilser3]

Description

Comparison of seg_ms_lesion_mp2rage and UNIseg, models for segment MS lesions on MP2RAGE images (UNIT1 contrast)

• Experiments performed using SCT v6.3
• To test both models, SC masks were generated using contrast-agnostic-softseg-spinalcord, then a cropping around the SC. Preprocessing pipeline for training and testing based on Try ensembling/bagging for prediction #63 (comment)

Preprocessing pipeline

subjects=( sub-P002    sub-P016        sub-P021        sub-P037...)

bids_basel_input="basel-mp2rage"

for subject in "${subjects[@]}"
    do
    # Set orient image and lesions mask to RPI 
    sct_image -i $bids_basel_input/$subject/anat/$subject"_UNIT1.nii.gz" -setorient RPI 
  
    # Generate SC with contrast agnostic model 
     sct_deepseg -i $bids_basel_input/$subject/anat/$subject"_UNIT1.nii.gz" -task seg_sc_contrast_agnostic -o $bids_basel_input/derivatives/labels/$subject/anat/$subject"_UNIT1_contrast-agnostic-SC_seg.nii.gz" 

    # Cropping with dilation (around the SC with 30 pixels in axial plane and 5 pixels in Z)
    sct_crop_image -i $bids_basel_input/$subject/anat/$subject"_UNIT1.nii.gz" -m $bids_basel_input/derivatives/labels/$subject/anat/$subject"_UNIT1_contrast-agnostic-SC_seg.nii.gz"  -o $bids_basel_input/$subject/anat/$subject"_UNIT1_crop.nii.gz"  -dilate 30x30x5

     # MS lesion segmentation using seg_ms_lesion_mp2rage
     sct_deepseg -i $bids_basel_input/$subject/anat/$subject"_UNIT1.nii.gz" -task seg_ms_lesion_mp2rage -o $bids_basel_input/derivatives/labels/$subject/anat/$subject"_UNIT1_crop_ivadomed_bin.nii.gz" 

     # Binarizing seg_ms_lesion_mp2rage soft seg 
     sct_maths -i $bids_basel_input/derivatives/labels/$subject/anat/$subject"_UNIT1_crop_ivadomed_seg.nii.gz" -bin 0.5 -o $bids_basel_input/derivatives/labels/$subject/anat/$subject"_UNIT1_crop_ivadomed_bin.nii.gz"

     # MS lesion segmentation using UNIseg
     sct_deepseg -i $bids_basel_input/$subject/anat/$subject"_UNIT1.nii.gz" -task UNIseg -o $bids_basel_input/derivatives/labels/$subject/anat/$subject"_UNIT1_UNIseg.nii.gz" 

done

Models

1. seg_ms_lesion_mp2rage (ivadomed)
   Current benchmark for segmenting MS lesions MP2RAGE. It was trained with ivadomed framework, using basel-mp2rage dataset, with a ensembling/bagging approaches. The soft segmentation will be binarized with a threshold of 0.5.
   It is implemented on sct_deepseg -task seg_ms_lesion_mp2rage

2. UNIseg
   Model trained with 3D nnUNetv2 framework, using basel-mp2rage, nih-ms-mp2rage and marseille-3t-mp2rage datasets.
   The best fold is locally implemented on sct_deepseg -task UNIseg

Details

Fig.01


Learning curves of the best fold of UNIseg training

Fig.02

Comparison of the Dice Score in few test subjects from different centers.

Fig.03

Comparison of the average inference time (seconds) of each model at different centers. Processing using a CPU (Intel(R) Core(TM) i3-1005G1 CPU @ 1.20GHz).

QC

Here the QC for test subjects (Fig.02)
Legend

• GT_lesion.nii.gz -> GT
• ivadomed_bin.nii.gz -> prediction of seg_ms_lesion_mp2rage with binarization at 0.5
• UNIseg.nii.gz -> prediction of UNIseg

Since UNIseg performs the same task on the same image modality, but with a higher performance (although a bit slower), we should replace seg_ms_lesion_mp2rage by UNIseg? creating a release?

Related issues

#75

[jcohenadad]

we should replace seg_ms_lesion_mp2rage by UNIseg? creating a release?

Agreed. Thank you for the thorough investigation @Nilser3. When doing the release you should link to this issue.