UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

The Impact of Using Voxel-Level Segmentation Metrics on Evaluating Multifocal Prostate Cancer Localisation

Yan, Wen; Yang, Qianye; Syer, Tom; Min, Zhe; Punwani, Shonit; Emberton, Mark; Barratt, Dean; ... Hu, Yipeng; + view all (2022) The Impact of Using Voxel-Level Segmentation Metrics on Evaluating Multifocal Prostate Cancer Localisation. In: Wu, S and Shabestari, B and Xing, L, (eds.) Applications of Medical Artificial Intelligence, AMAI 2022. (pp. pp. 128-138). Springer Nature Switzerland: Cham, Switzerland. Green open access

[thumbnail of 2203.16415v2.pdf]
Preview
Text
2203.16415v2.pdf - Accepted Version

Download (5MB) | Preview

Abstract

Dice similarity coefficient (DSC) and Hausdorff distance (HD) are widely used for evaluating medical image segmentation. They have also been criticised, when reported alone, for their unclear or even misleading clinical interpretation. DSCs may also differ substantially from HDs, due to boundary smoothness or multiple regions of interest (ROIs) within a subject. More importantly, either metric can also have a nonlinear, non-monotonic relationship with outcomes based on Type 1 and 2 errors, designed for specific clinical decisions that use the resulting segmentation. Whilst cases causing disagreement between these metrics are not difficult to postulate, one might argue that they may not necessarily be substantiated in real-world segmentation applications, as a majority of ROIs and their predictions often do not manifest themselves in extremely irregular shapes or locations that are prone to such inconsistency. This work first proposes a new asymmetric detection metric, adapting those used in object detection, for planning prostate cancer procedures. The lesion-level metrics is then compared with the voxel-level DSC and HD, whereas a 3D UNet is used for segmenting lesions from multiparametric MR (mpMR) images. Based on experimental results using 877 sets of mpMR images, we report pairwise agreement and correlation 1) between DSC and HD, and 2) between voxel-level DSC and recall-controlled precision at lesion-level, with Cohen’s κ∈ [ 0.49, 0.61 ] and Pearson’s r∈ [ 0.66, 0.76 ] (p-values<0.001) at varying cut-offs. However, the differences in false-positives and false-negatives, between the actual errors and the perceived counterparts if DSC is used, can be as high as 152 and 154, respectively, out of the 357 test set lesions. We therefore carefully conclude that, despite of the significant correlations, voxel-level metrics such as DSC can misrepresent lesion-level detection accuracy for evaluating localisation of multifocal prostate cancer and should be interpreted with caution.

Type: Proceedings paper
Title: The Impact of Using Voxel-Level Segmentation Metrics on Evaluating Multifocal Prostate Cancer Localisation
Event: International Workshop on Applications of Medical AI - AMAI 2022
Location: Singapore, SINGAPORE
Dates: 18 Sep 2022
Open access status: An open access version is available from UCL Discovery
DOI: 10.1007/978-3-031-17721-7_14
Publisher version: https://doi.org/10.1007/978-3-031-17721-7_14
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Science & Technology, Technology, Life Sciences & Biomedicine, Computer Science, Artificial Intelligence, Medical Informatics, Computer Science, Prostate cancer, Multi-parametric MR, Lesion-level localisation metrics, Voxel-level segmentation metrics, PARAMETRIC MRI, BIOPSY, ACCURACY, PROMIS
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences > Div of Medicine
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Med Phys and Biomedical Eng
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences > Div of Medicine > Department of Imaging
URI: https://discovery.ucl.ac.uk/id/eprint/10161434
Downloads since deposit
4Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item