Abstract
When dealing with Deep Learning applications in open-set problems, accurately classifying known classes seen in the training phase is not the only aspect to be taken into account. In such a context, detecting Out-of-Distribution (OOD) samples plays an important role as an auxiliary task, generally solved by OOD detection methods. For medical applications, detecting unknown samples may in classification problems can be beneficial for many aspects, such as a better understanding of the diagnosis and probably a more adequate treatment. In this article, we evaluate a feature space-based approach, named as OpenPCS-Class, for OOD detection in medical applications, more specifically skin lesion classification. We compare the OpenPCS-Class against important OOD detection methods, evaluating different model architectures and OOD datasets. The OpenPCS-Class outperformed other methods at 48.4% and 5.3% in terms of FPR95 and AUROC, respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Code available at https://github.com/mdrs-thiago/skin-lesion-ood-detection.
References
Ali, S.N., et al.: Monkeypox skin lesion detection using deep learning models: a preliminary feasibility study. arXiv preprint arXiv:2207.03342 (2022)
Berger, C., Paschali, M., Glocker, B., Kamnitsas, K.: Confidence-based out-of-distribution detection: a comparative study and analysis. In: Sudre, C.H., et al. (eds.) UNSURE/PIPPI -2021. LNCS, vol. 12959, pp. 122–132. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87735-4_12
Calderon-Ramirez, S., Yang, S., Elizondo, D., Moemeni, A.: Dealing with distribution mismatch in semi-supervised deep learning for COVID-19 detection using chest X-ray images: a novel approach using feature densities. Appl. Soft Comput. 123, 108983 (2022)
Cao, T., Huang, C.W., Hui, D.Y.T., Cohen, J.P.: A benchmark of medical out of distribution detection. arXiv preprint arXiv:2007.04250 (2020)
Carvalho, T., Vellasco, M., Amaral, J.F.: Out-of-distribution detection in deep learning models: a feature space-based approach. In: International Joint Conference on Neural Networks (2023)
Cho, W., Park, J., Choo, J.: Training auxiliary prototypical classifiers for explainable anomaly detection in medical image segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2624–2633 (2023)
Dosovitskiy, A., et al.: An image is worth 16 \(\times \)16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hendrycks, D., et al.: Scaling out-of-distribution detection for real-world settings. In: International Conference on Machine Learning, pp. 8759–8773. PMLR (2022)
Hendrycks, D., Carlini, N., Schulman, J., Steinhardt, J.: Unsolved problems in ML safety. arXiv preprint arXiv:2109.13916 (2021)
Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136 (2016)
Kamoi, R., Kobayashi, K.: Why is the mahalanobis distance effective for anomaly detection? arXiv preprint arXiv:2003.00402 (2020)
Karimi, D., Gholipour, A.: Improving calibration and out-of-distribution detection in deep models for medical image segmentation. IEEE Trans. Artif. Intell. 4, 383–397 (2022)
Lambert, B., Forbes, F., Doyle, S., Tucholka, A., Dojat, M.: Improving uncertainty-based out-of-distribution detection for medical image segmentation. arXiv preprint arXiv:2211.05421 (2022)
Lee, K., Lee, K., Lee, H., Shin, J.: A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In: Advances in neural information processing systems, vol. 31 (2018)
Liu, W., Wang, X., Owens, J., Li, Y.: Energy-based out-of-distribution detection. Adv. Neural. Inf. Process. Syst. 33, 21464–21475 (2020)
Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., Xie, S.: A convnet for the 2020s. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11976–11986 (2022)
Martinez, J.A.C., Oliveira, H., dos Santos, J.A., Feitosa, R.Q.: Open set semantic segmentation for multitemporal crop recognition. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2021)
Muhammad, K., et al.: Vision-based semantic segmentation in scene understanding for autonomous driving: recent achievements, challenges, and outlooks. IEEE Trans. Intell. Transp. Syst. 23, 22694–22715 (2022)
Nunes, I., Pereira, M.B., Oliveira, H., Santos, J.A.D., Poggi, M.: Fuss: Fusing superpixels for improved segmentation consistency. arXiv preprint arXiv:2206.02714 (2022)
Oliveira, H., Silva, C., Machado, G.L., Nogueira, K., Dos Santos, J.A.: Fully convolutional open set segmentation. Mach. Learn. 112, 1733–1784 (2021)
Pawlowski, N., Glocker, B.: Abnormality detection in histopathology via density estimation with normalising flows. In: Medical Imaging with Deep Learning (2021)
Podolskiy, A., Lipin, D., Bout, A., Artemova, E., Piontkovskaya, I.: Revisiting Mahalanobis distance for transformer-based out-of-domain detection. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 13675–13682 (2021)
Ren, J., Fort, S., Liu, J., Roy, A.G., Padhy, S., Lakshminarayanan, B.: A simple fix to mahalanobis distance for improving near-OOD detection. arXiv preprint arXiv:2106.09022 (2021)
Roy, A.G., et al.: Does your dermatology classifier know what it doesn’t know? detecting the long-tail of unseen conditions. Med. Image Anal. 75, 102274 (2022)
Swetha, P., Srilatha, J.: Applications of speech recognition in the agriculture sector: a review. ECS Trans. 107(1), 19377 (2022)
Tschandl, P., Rosendahl, C., Kittler, H.: The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5(1), 1–9 (2018)
Uwimana, A., Senanayake, R.: Out of distribution detection and adversarial attacks on deep neural networks for robust medical image analysis. arXiv preprint arXiv:2107.04882 (2021)
Wightman, R., Touvron, H., Jégou, H.: ResNet strikes back: an improved training procedure in timm. arXiv preprint arXiv:2110.00476 (2021)
Wollek, A., Willem, T., Ingrisch, M., Sabel, B., Lasser, T.: A knee cannot have lung disease: out-of-distribution detection with in-distribution voting using the medical example of chest X-ray classification. arXiv preprint arXiv:2208.01077 (2022)
Wright, J., Ma, Y.: High-Dimensional Data Analysis with Low-Dimensional Models: Principles, Computation, and Applications. Cambridge University Press (2022)
Wu, Y., et al.: Revisit overconfidence for OOD detection: reassigned contrastive learning with adaptive class-dependent threshold. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4165–4179 (2022)
Wu, Y., et al.: Disentangling confidence score distribution for out-of-domain intent detection with energy-based learning. arXiv preprint arXiv:2210.08830 (2022)
Yang, J., Zhou, K., Li, Y., Liu, Z.: Generalized out-of-distribution detection: a survey. arXiv preprint arXiv:2110.11334 (2021)
Ye, N., et al.: OOD-bench: quantifying and understanding two dimensions of out-of-distribution generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7947–7958 (2022)
Zadorozhny, K., Thoral, P., Elbers, P., Ciná, G.: Out-of-distribution detection for medical applications: guidelines for practical evaluation. In: Shaban-Nejad, A., Michalowski, M., Bianco, S. (eds.) Multimodal AI in Healthcare. Studies in Computational Intelligence, vol. 1060, pp. 137–153. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-14771-5_10
Zhang, O., Delbrouck, J.-B., Rubin, D.L.: Out of distribution detection for medical images. In: Sudre, C.H., et al. (eds.) UNSURE/PIPPI -2021. LNCS, vol. 12959, pp. 102–111. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87735-4_10
Zimmerman, D.W., Zumbo, B.D.: Rank transformations and the power of the student T test and welch T’test for non-normal populations with unequal variances. Can. J. Exp. Psychol. 47(3), 523 (1993)
Acknowledgements
This work was supported in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001, Conselho Nacional de Desenvolvimento e Pesquisa (CNPq) under Grants 140254/2021-8 and 308717/2020-1, and Fundação de Amparo à Pesquisa do Rio de Janeiro (FAPERJ)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Carvalho, T., Vellasco, M., Amaral, J.F., Figueiredo, K. (2023). A Feature-Based Out-of-Distribution Detection Approach in Skin Lesion Classification. In: Naldi, M.C., Bianchi, R.A.C. (eds) Intelligent Systems. BRACIS 2023. Lecture Notes in Computer Science(), vol 14196. Springer, Cham. https://doi.org/10.1007/978-3-031-45389-2_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-45389-2_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45388-5
Online ISBN: 978-3-031-45389-2
eBook Packages: Computer ScienceComputer Science (R0)