Distributional Gaussian Processes Layers for Out-of-Distribution Detection

Sebastian G. Popescu1, David J. Sharp1, James H. Cole2, Konstantinos Kamnitsas1, Ben Glocker1
1: Imperial College London, 2: University College London
Publication date: 2022/06/29
PDF · Code · Video · arXiv


Machine learning models deployed on medical imaging tasks must be equipped with out-of-distribution detection capabilities in order to avoid erroneous predictions. It is unsure whether out-of-distribution detection models reliant on deep neural networks are suitable for detecting domain shifts in medical imaging. Gaussian Processes can reliably separate in-distribution data points from out-of-distribution data points via their mathematical construction. Hence, we propose a parameter efficient Bayesian layer for hierarchical convolutional Gaussian Processes that incorporates Gaussian Processes operating in Wasserstein-2 space to reliably propagate uncertainty. This directly replaces convolving Gaussian Processes with a distance-preserving affine operator on distributions. Our experiments on brain tissue-segmentation show that the resulting architecture approaches the performance of well-established deterministic segmentation algorithms (U-Net), which has not been achieved with previous hierarchical Gaussian Processes. Moreover, by applying the same segmentation model to out-of-distribution data (i.e., images with pathology such as brain tumors), we show that our uncertainty estimates result in out-of-distribution detection that outperforms the capabilities of previous Bayesian networks and reconstruction-based approaches that learn normative distributions.


gaussian processes · image segmentation · out-of-distribution detection

Bibtex @article{melba:2022:009:popescu, title = "Distributional Gaussian Processes Layers for Out-of-Distribution Detection", author = "Popescu, Sebastian G. and Sharp, David J. and Cole, James H. and Kamnitsas, Konstantinos and Glocker, Ben", journal = "Machine Learning for Biomedical Imaging", volume = "1", issue = "IPMI 2021 special issue", year = "2022", pages = "1--64", issn = "2766-905X", doi = "https://doi.org/10.59275/j.melba.2022-a6bf", url = "https://melba-journal.org/2022:009" }
RISTY - JOUR AU - Popescu, Sebastian G. AU - Sharp, David J. AU - Cole, James H. AU - Kamnitsas, Konstantinos AU - Glocker, Ben PY - 2022 TI - Distributional Gaussian Processes Layers for Out-of-Distribution Detection T2 - Machine Learning for Biomedical Imaging VL - 1 IS - IPMI 2021 special issue SP - 1 EP - 64 SN - 2766-905X DO - https://doi.org/10.59275/j.melba.2022-a6bf UR - https://melba-journal.org/2022:009 ER -

2022:009 cover