Modeling Annotation Uncertainty with Gaussian Heatmaps in Landmark Localization

Franz ThalerMedical University of Graz, Austria, Christian PayerGraz University of Technology, Austria, Martin UrschlerThe University of Auckland, New Zealand, Darko ŠternMedical University of Graz, Austria
UNSURE2020 special issue
Publication date: 2021/09/24
PDF · arXiv · Video · Code


In landmark localization, due to ambiguities in defining their exact position, landmark annotations may suffer from large observer variabilities, which result in uncertain annotations. To model the annotation ambiguities of the training dataset, we propose to learn anisotropic Gaussian parameters modeling the shape of the target heatmap during optimization. Furthermore, our method models the prediction uncertainty of individual samples by fitting anisotropic Gaussian functions to the predicted heatmaps during inference. Besides state-of-the-art results, our experiments on datasets of hand radiographs and lateral cephalograms also show that Gaussian functions are correlated with both localization accuracy and observer variability. As a final experiment, we show the importance of integrating the uncertainty into decision making by measuring the influence of the predicted location uncertainty on the classification of anatomical abnormalities in lateral cephalograms.


inter-observer variability · fully-convolutional neural network · heatmap regression · uncertainty estimation · landmark localization

Bibtex @article{melba:2021:014:thaler, title = "Modeling Annotation Uncertainty with Gaussian Heatmaps in Landmark Localization", authors = "Thaler, Franz and Payer, Christian and Urschler, Martin and Štern, Darko", journal = "Machine Learning for Biomedical Imaging", volume = "1", issue = "UNSURE2020 special issue", year = "2021" }

2021:014 cover