Focused Decoding Enables 3D Anatomical Detection by Transformers

Bastian Wittmann1, Fernando Navarro2, Suprosanna Shit2, Bjoern Menze1
1: Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland, 2: Department of Informatics, Technical University of Munich, Munich, Germany
Publication date: 2023/02/27
PDF · Code · arXiv


Detection Transformers represent end-to-end object detection approaches based on a Transformer encoder-decoder architecture, exploiting the attention mechanism for global relation modeling. Although Detection Transformers deliver results on par with or even superior to their highly optimized CNN-based counterparts operating on 2D natural images, their success is closely coupled to access to a vast amount of training data. This, however, restricts the feasibility of employing Detection Transformers in the medical domain, as access to annotated data is typically limited. To tackle this issue and facilitate the advent of medical Detection Transformers, we propose a novel Detection Transformer for 3D anatomical structure detection, dubbed Focused Decoder. Focused Decoder leverages information from an anatomical region atlas to simultaneously deploy query anchors and restrict the crossattention’s field of view to regions of interest, which allows for a precise focus on relevant anatomical structures. We evaluate our proposed approach on two publicly available CT datasets and demonstrate that Focused Decoder not only provides strong detection results and thus alleviates the need for a vast amount of annotated data but also exhibits exceptional and highly intuitive explainability of results via attention weights. Our code is available at


Anatomical Structure Localization · Attention · Computed Tomography · Deep Learning · Detection Transformer · Explainable AI

Bibtex @article{melba:2023:003:wittmann, title = "Focused Decoding Enables 3D Anatomical Detection by Transformers", author = "Wittmann, Bastian and Navarro, Fernando and Shit, Suprosanna and Menze, Bjoern", journal = "Machine Learning for Biomedical Imaging", volume = "2", issue = "February 2023 issue", year = "2023", pages = "72--95", issn = "2766-905X", doi = "", url = "" }
RISTY - JOUR AU - Wittmann, Bastian AU - Navarro, Fernando AU - Shit, Suprosanna AU - Menze, Bjoern PY - 2023 TI - Focused Decoding Enables 3D Anatomical Detection by Transformers T2 - Machine Learning for Biomedical Imaging VL - 2 IS - February 2023 issue SP - 72 EP - 95 SN - 2766-905X DO - UR - ER -

2023:003 cover