Evaluation of 3D GANs for Lung Tissue Modelling in Pulmonary CT

Sam Ellis1, Octavio Martinez Manzanera1, Vasileios Baltatzis1,2, Ibrahim Nawaz3, Arjun Nair4, Loic Le Folgoc2, Sujal Desai5, Ben Glocker2, Julia A Schnabel1,6,7
1: King’s College London, UK, 2: Imperial College London, UK, 3: King’s College London, 4: University College London, UK, 5: The Royal Brompton & Harefield NHS Foundation Trust, UK, 6: Helmholtz Center Munich, Germany, 7: Technical University of Munich, Germany
August 2022 issue
Publication date: 2022/08/18
PDF · arXiv · Code

Abstract

Generative adversarial networks (GANs) are able to model accurately the distribution of complex, high-dimensional datasets, for example images. This characteristic makes high-quality GANs useful for unsupervised anomaly detection in medical imaging. However, differences in training datasets such as output image dimensionality and appearance of semantically meaningful features mean that GAN models from the natural image processing domain may not work 'out-of-the-box' for medical imaging applications, necessitating re-implementation and re-evaluation. In this work we adapt and evaluate three GAN models to the application of modelling 3D healthy image patches for pulmonary CT. To the best of our knowledge, this is the first time that such a detailed evaluation has been performed. The deep convolutional GAN (DCGAN), styleGAN and the bigGAN architectures were selected for investigation due to their ubiquity and high performance in natural image processing. We train different variants of these methods and assess their performance using the widely used Frechet Inception Distance (FID). In addition, the quality of the generated images was evaluated by a human observer study, the ability of the networks to model 3D domain-specific features was investigated, and the structure of the GAN latent spaces was analysed. Results show that the 3D styleGAN approaches produce realistic-looking images with meaningful 3D structure, but suffer from mode collapse which must be explicitly addressed during training to obtain diversity in the samples. Conversely, the 3D DCGAN models show a greater capacity for image variability, but at the cost of poor-quality images. The 3D bigGAN models provide an intermediate level of image quality, but most accurately model the distribution of selected semantically meaningful features. The results suggest that future development is required to realise a 3D GAN with sufficient representational capacity for patch-based lung CT anomaly detection and we offer recommendations for future areas of research, such as experimenting with other architectures and incorporation of position-encoding.

Keywords

Pulmonary CT · Generative modelling · GANs

Bibtex @article{melba:2022:024:ellis, title = "Evaluation of 3D GANs for Lung Tissue Modelling in Pulmonary CT", author = "Ellis, Sam and Martinez Manzanera, Octavio and Baltatzis, Vasileios and Nawaz, Ibrahim and Nair, Arjun and Le Folgoc, Loic and Desai, Sujal and Glocker, Ben and Schnabel, Julia A", journal = "Machine Learning for Biomedical Imaging", volume = "1", issue = "August 2022 issue", year = "2022", issn = "2766-905X", url = "https://melba-journal.org/papers/2022:024.html" }
RISTY - JOUR AU - Ellis, Sam AU - Martinez Manzanera, Octavio AU - Baltatzis, Vasileios AU - Nawaz, Ibrahim AU - Nair, Arjun AU - Le Folgoc, Loic AU - Desai, Sujal AU - Glocker, Ben AU - Schnabel, Julia A PY - 2022 TI - Evaluation of 3D GANs for Lung Tissue Modelling in Pulmonary CT T2 - Machine Learning for Biomedical Imaging VL - 1 IS - August 2022 issue SN - 2766-905X UR - https://www.melba-journal.org/papers/2022:024.html ER -

2022:024 cover