Ju, Lie; Zhou, Sijin; Zhou, Yukun; Lu, Huimin; Zhu, Zhuoting; Keane, Pearse A; Ge, Zongyuan; (2025) Delving Into Out-of-Distribution Detection with Medical Vision-Language Models. In: Gee, James C and Alexander, Daniel C and Hong, Jaesung and Iglesias, Juan Eugenio and Sudre, Carole H and Venkataraman, Archana and Golland, Polina and Kim, Jong Hyo and Park, Jinah, (eds.) Lecture Notes in Computer Science. (pp. pp. 133-143). Springer Nature Switzerland (In press).

Text VLM_OOD.pdf - Accepted Version Access restricted to UCL open access staff until 21 September 2026. Download (1MB)

Abstract

Recent advances in medical vision-language models (VLMs) demonstrate impressive performance in image classification tasks, driven by their strong zero-shot generalization capabilities. However, given the high variability and complexity inherent in medical imaging data, the ability of these models to detect out-of-distribution (OOD) data in this domain remains underexplored. In this work, we conduct the first systematic investigation into the OOD detection potential of medical VLMs. We evaluate state-of-the-art VLM-based OOD detection methods across a diverse set of medical VLMs, including both general and domain-specific purposes. To accurately reflect real-world challenges, we introduce a cross-modality evaluation pipeline for benchmarking full-spectrum OOD detection, rigorously assessing model robustness against both semantic shifts and covariate shifts. Furthermore, we propose a novel hierarchical prompt-based method that significantly enhances OOD detection performance. Extensive experiments are conducted to validate the effectiveness of our approach. The codes are available at https://github.com/PyJulie/Medical-VLMs-OOD-Detection

Download activity - last month

Export as XMLCSVJSON

Download activity - last 12 months

Export as XMLCSVJSON

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item