Joint Frequency and Image Space Learning for MRI Reconstruction and Analysis

Nalini M. Singh10000-0003-3584-2198, Juan Eugenio Iglesias2,3,4,10000-0001-7569-173X, Elfar Adalsteinsson10000-0002-7637-2914, Adrian V. Dalca2,3,10000-0002-8422-0136, Polina Golland10000-0003-2516-731X
1: Massachusetts Institute of Technology, 2: Massachusetts General Hospital, 3: Harvard Medical School, 4: University College London
Publication date: 2022/06/23
PDF · Code · Video · arXiv


We propose neural network layers that explicitly combine frequency and image feature representations and show that they can be used as a versatile building block for reconstruction from frequency space data. Our work is motivated by the challenges arising in MRI acquisition where the signal is a corrupted Fourier transform of the desired image. The proposed joint learning schemes enable both correction of artifacts native to the frequency space and manipulation of image space representations to reconstruct coherent image structures at every layer of the network. This is in contrast to most current deep learning approaches for image reconstruction that treat frequency and image space features separately and often operate exclusively in one of the two spaces. We demonstrate the advantages of joint convolutional learning for a variety of tasks, including motion correction, denoising, reconstruction from undersampled acquisitions, and combined undersampling and motion correction on simulated and real world multicoil MRI data. The joint models produce consistently high quality output images across all tasks and datasets. When integrated into a state of the art unrolled optimization network with physics-inspired data consistency constraints for undersampled reconstruction, the proposed architectures significantly improve the optimization landscape, which yields an order of magnitude reduction of training time. This result suggests that joint representations are particularly well suited for MRI signals in deep learning networks. Our code and pretrained models are publicly available at


Magnetic Resonance Imaging · Deep Learning · Undersampled Reconstruction · Motion Correction · Denoising

Bibtex @article{melba:2022:018:singh, title = "Joint Frequency and Image Space Learning for MRI Reconstruction and Analysis", author = "Singh, Nalini M. and Iglesias, Juan Eugenio and Adalsteinsson, Elfar and Dalca, Adrian V. and Golland, Polina", journal = "Machine Learning for Biomedical Imaging", volume = "1", issue = "June 2022 issue", year = "2022", pages = "1--28", issn = "2766-905X", doi = "", url = "" }
RISTY - JOUR AU - Singh, Nalini M. AU - Iglesias, Juan Eugenio AU - Adalsteinsson, Elfar AU - Dalca, Adrian V. AU - Golland, Polina PY - 2022 TI - Joint Frequency and Image Space Learning for MRI Reconstruction and Analysis T2 - Machine Learning for Biomedical Imaging VL - 1 IS - June 2022 issue SP - 1 EP - 28 SN - 2766-905X DO - UR - ER -

2022:018 cover