Distributional Gaussian Processes Layers for Out-of-Distribution Detection

Sebastian G. PopescuImperial College London, David J. SharpImperial College London, James H. ColeUniversity College London, Konstantinos KamnitsasImperial College London, Ben GlockerImperial College London
IPMI 2021 special issue
Publication date: 2022/06/29
PDF · arXiv · Code · Video

Abstract

Machine learning models deployed on medical imaging tasks must be equipped with out-of-distribution detection capabilities in order to avoid erroneous predictions. It is unsure whether out-of-distribution detection models reliant on deep neural networks are suitable for detecting domain shifts in medical imaging. Gaussian Processes can reliably separate in-distribution data points from out-of-distribution data points via their mathematical construction. Hence, we propose a parameter efficient Bayesian layer for hierarchical convolutional Gaussian Processes that incorporates Gaussian Processes operating in Wasserstein-2 space to reliably propagate uncertainty. This directly replaces convolving Gaussian Processes with a distance-preserving affine operator on distributions. Our experiments on brain tissue-segmentation show that the resulting architecture approaches the performance of well-established deterministic segmentation algorithms (U-Net), which has not been achieved with previous hierarchical Gaussian Processes. Moreover, by applying the same segmentation model to out-of-distribution data (i.e., images with pathology such as brain tumors), we show that our uncertainty estimates result in out-of-distribution detection that outperforms the capabilities of previous Bayesian networks and reconstruction-based approaches that learn normative distributions.

Keywords

gaussian processes · image segmentation · out-of-distribution detection

Bibtex@article{melba:2022:009:popescu, title = "Distributional Gaussian Processes Layers for Out-of-Distribution Detection", authors = "Popescu, Sebastian G. and Sharp, David J. and Cole, James H. and Kamnitsas, Konstantinos and Glocker, Ben", journal = "Machine Learning for Biomedical Imaging", volume = "1", issue = "IPMI 2021 special issue", year = "2022" }

2022:009 cover