Leveraging CAM Algorithms for Explaining Medical Semantic Segmentation

Tillmann Rheude1,2, Andreas Wirtz2, Arjan Kuijper1,2, Stefan Wesarg2
1: TU Darmstadt, Darmstadt, Germany, 2: Fraunhofer Institute for Computer Graphics Research (IGD), Darmstadt, Germany
Publication date: 2024/10/01
https://doi.org/10.59275/j.melba.2024-ebd3
PDF · Code

Abstract

Convolutional neural networks (CNNs) achieve prevailing results in segmentation tasks nowadays and represent the state-of-the-art for image-based analysis. However, the under- standing of the accurate decision-making process of a CNN is rather unknown. The research area of explainable artificial intelligence (xAI) primarily revolves around understanding and interpreting this black-box behavior. One way of interpreting a CNN is the use of class ac- tivation maps (CAMs) that represent heatmaps to indicate the importance of image areas for the prediction of the CNN. For classification tasks, a variety of CAM algorithms exist. But for segmentation tasks, only one CAM algorithm for the interpretation of the output of a CNN exist. We propose a transfer between existing classification- and segmentation- based methods for more detailed, explainable, and consistent results which show salient pixels in semantic segmentation tasks. The resulting Seg-HiRes-Grad CAM is an exten- sion of the segmentation-based Seg-Grad CAM with the transfer to the classification-based HiRes CAM. Our method improves the previously-mentioned existing segmentation-based method by adjusting it to recently published classification-based methods. Especially for medical image segmentation, this transfer solves existing explainability disadvantages. The code is available at https://github.com/TillmannRheude/SegHiResGrad_CAM

Keywords

Deep Learning · Explainable Artificial Intelligence · Gradient-Based Methods · Medical Image Segmentation

Bibtex @article{melba:2024:023:rheude, title = "Leveraging CAM Algorithms for Explaining Medical Semantic Segmentation", author = "Rheude, Tillmann and Wirtz, Andreas and Kuijper, Arjan and Wesarg, Stefan", journal = "Machine Learning for Biomedical Imaging", volume = "2", issue = "iMIMIC 2023 special issue", year = "2024", pages = "2089--2102", issn = "2766-905X", doi = "https://doi.org/10.59275/j.melba.2024-ebd3", url = "https://melba-journal.org/2024:023" }
RISTY - JOUR AU - Rheude, Tillmann AU - Wirtz, Andreas AU - Kuijper, Arjan AU - Wesarg, Stefan PY - 2024 TI - Leveraging CAM Algorithms for Explaining Medical Semantic Segmentation T2 - Machine Learning for Biomedical Imaging VL - 2 IS - iMIMIC 2023 special issue SP - 2089 EP - 2102 SN - 2766-905X DO - https://doi.org/10.59275/j.melba.2024-ebd3 UR - https://melba-journal.org/2024:023 ER -

2024:023 cover