Joint Frequency and Image Space Learning for MRI Reconstruction and Analysis

Nalini M. SinghMassachusetts Institute of Technology, Juan Eugenio IglesiasMassachusetts General Hospital
Harvard Medical School
University College London
Massachusetts Institute of Technology
, Elfar AdalsteinssonMassachusetts Institute of Technology, Adrian V. DalcaMassachusetts General Hospital
Harvard Medical School
Massachusetts Institute of Technology
, Polina GollandMassachusetts Institute of Technology
June 2022 issue
Publication date: 2022/06/23
PDF · arXiv · Code · Video


We propose neural network layers that explicitly combine frequency and image feature representations and show that they can be used as a versatile building block for reconstruction from frequency space data. Our work is motivated by the challenges arising in MRI acquisition where the signal is a corrupted Fourier transform of the desired image. The proposed joint learning schemes enable both correction of artifacts native to the frequency space and manipulation of image space representations to reconstruct coherent image structures at every layer of the network. This is in contrast to most current deep learning approaches for image reconstruction that treat frequency and image space features separately and often operate exclusively in one of the two spaces. We demonstrate the advantages of joint convolutional learning for a variety of tasks, including motion correction, denoising, reconstruction from undersampled acquisitions, and combined undersampling and motion correction on simulated and real world multicoil MRI data. The joint models produce consistently high quality output images across all tasks and datasets. When integrated into a state of the art unrolled optimization network with physics-inspired data consistency constraints for undersampled reconstruction, the proposed architectures significantly improve the optimization landscape, which yields an order of magnitude reduction of training time. This result suggests that joint representations are particularly well suited for MRI signals in deep learning networks. Our code and pretrained models are publicly available at


Magnetic Resonance Imaging · Deep Learning · Undersampled Reconstruction · Motion Correction · Denoising

Bibtex @article{melba:2022:018:singh, title = "Joint Frequency and Image Space Learning for MRI Reconstruction and Analysis", authors = "Singh, Nalini M. and Iglesias, Juan Eugenio and Adalsteinsson, Elfar and Dalca, Adrian V. and Golland, Polina", journal = "Machine Learning for Biomedical Imaging", volume = "1", issue = "June 2022 issue", year = "2022" }

2022:018 cover