Distributional Gaussian Processes Layers for Out-of-Distribution Detection

Sebastian G. Popescu1, David J. Sharp1, James H. Cole2, Konstantinos Kamnitsas1, Ben Glocker1
1: Imperial College London, 2: University College London
IPMI 2021 special issue
Publication date: 2022/06/29
PDF · arXiv · Code · Video

Abstract

Machine learning models deployed on medical imaging tasks must be equipped with out-of-distribution detection capabilities in order to avoid erroneous predictions. It is unsure whether out-of-distribution detection models reliant on deep neural networks are suitable for detecting domain shifts in medical imaging. Gaussian Processes can reliably separate in-distribution data points from out-of-distribution data points via their mathematical construction. Hence, we propose a parameter efficient Bayesian layer for hierarchical convolutional Gaussian Processes that incorporates Gaussian Processes operating in Wasserstein-2 space to reliably propagate uncertainty. This directly replaces convolving Gaussian Processes with a distance-preserving affine operator on distributions. Our experiments on brain tissue-segmentation show that the resulting architecture approaches the performance of well-established deterministic segmentation algorithms (U-Net), which has not been achieved with previous hierarchical Gaussian Processes. Moreover, by applying the same segmentation model to out-of-distribution data (i.e., images with pathology such as brain tumors), we show that our uncertainty estimates result in out-of-distribution detection that outperforms the capabilities of previous Bayesian networks and reconstruction-based approaches that learn normative distributions.

Keywords

gaussian processes · image segmentation · out-of-distribution detection

Bibtex @article{melba:2022:009:popescu, title = "Distributional Gaussian Processes Layers for Out-of-Distribution Detection", author = "Popescu, Sebastian G. and Sharp, David J. and Cole, James H. and Kamnitsas, Konstantinos and Glocker, Ben", journal = "Machine Learning for Biomedical Imaging", volume = "1", issue = "IPMI 2021 special issue", year = "2022", issn = "2766-905X", url = "https://melba-journal.org/papers/2022:009.html" }
RISTY - JOUR AU - Popescu, Sebastian G. AU - Sharp, David J. AU - Cole, James H. AU - Kamnitsas, Konstantinos AU - Glocker, Ben PY - 2022 TI - Distributional Gaussian Processes Layers for Out-of-Distribution Detection T2 - Machine Learning for Biomedical Imaging VL - 1 IS - IPMI 2021 special issue SN - 2766-905X UR - https://www.melba-journal.org/papers/2022:009.html ER -

2022:009 cover