The Alzheimer's Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge: Results after 1 Year Follow-up

Razvan V. MarinescuCentre for Medical Image Computing, University College London, UK
Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, USA
, Neil P. OxtobyCentre for Medical Image Computing, University College London, UK, Alexandra L. YoungCentre for Medical Image Computing, University College London, UK
Department of Neuroimaging, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
, Esther E. BronBiomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Netherlands, Arthur W. TogaLaboratory of NeuroImaging, University of Southern California, USA, Michael W. WeinerCenter for Imaging of Neurodegenerative Diseases, University of California San Francisco, USA, Frederik BarkhofCentre for Medical Image Computing, University College London, UK
Department of Radiology and Nuclear Medicine, VU Medical Centre, Netherlands
Dementia Research Centre and the UK Dementia Research Institute, UCL Queen Square Institute of Neurology, UK
, Nick C. FoxDementia Research Centre and the UK Dementia Research Institute, UCL Queen Square Institute of Neurology, UK, Arman EshaghiQueen Square Multiple Sclerosis Centre, UCL Queen Square Institute of Neurology, UK
Centre for Medical Image Computing, University College London, UK
, Tina ToniAuthor not affiliated with any research institution, Marcin SalaterskiAuthor not affiliated with any research institution, Veronika LuninaAuthor not affiliated with any research institution, Manon AnsartInstitut du Cerveau et de la Moelle épinière, Paris, France, Stanley DurrlemanInstitut du Cerveau et de la Moelle épinière, Paris, France, Pascal LuInstitut du Cerveau et de la Moelle épinière, Paris, France, Samuel IddiAlzheimer's Therapeutic Research Institute, University of Southern California, USA
Department of Statistics and Actuarial Science, University of Ghana, Ghana
, Dan LiAlzheimer's Therapeutic Research Institute, University of Southern California, USA, Wesley K. ThompsonDepartment of Family Medicine and Public Health, University of California San Diego, USA, Michael C. DonohueAlzheimer's Therapeutic Research Institute, University of Southern California, USA, Aviv NahonBen Gurion University of the Negev, Beersheba, Israel, Yarden LevyBen Gurion University of the Negev, Beersheba, Israel, Dan HalbersbergBen Gurion University of the Negev, Beersheba, Israel, Mariya CohenBen Gurion University of the Negev, Beersheba, Israel, Huiling LiaoThe University of Texas Health Science Center at Houston, Houston, USA, Tengfei LiThe University of Texas Health Science Center at Houston, Houston, USA, Kaixian YuThe University of Texas Health Science Center at Houston, Houston, USA, Hongtu ZhuThe University of Texas Health Science Center at Houston, Houston, USA, José G. Tamez-PeñaInstituto Tecnológico y de Estudios Superiores de Monterrey, Monterrey, Mexico, Aya IsmailUniversity of Maryland, College Park, USA, Timothy WoodUniversity of Maryland, College Park, USA, Hector Corrada BravoUniversity of Maryland, College Park, USA, Minh NguyenNational University of Singapore, Singapore, Singapore, Nanbo SunNational University of Singapore, Singapore, Singapore, Jiashi FengNational University of Singapore, Singapore, Singapore, B.T. Thomas YeoNational University of Singapore, Singapore, Singapore, Gang ChenMedical College of Wisconsin, Milwaukee, USA, Ke QiEmory University, Atlanta, USA, Shiyang ChenEmory University, Atlanta, USA
Georgia Institute of Technology, Atlanta, USA
, Deqiang QiuEmory University, Atlanta, USA
Georgia Institute of Technology, Atlanta, USA
, Ionut BuciumanVasile Lucaciu National College, Baia Mare, Romania, Alex KelnerVasile Lucaciu National College, Baia Mare, Romania, Raluca PopVasile Lucaciu National College, Baia Mare, Romania, Denisa RimoceaVasile Lucaciu National College, Baia Mare, Romania, Mostafa M. GhaziBiomediq A/S, Denmark
Cerebriu A/S, Denmark
University of Copenhagen, Denmark
Centre for Medical Image Computing, University College London, UK
, Mads NielsenBiomediq A/S, Denmark
Cerebriu A/S, Denmark
University of Copenhagen, Denmark
, Sebastien OurselinSchool of Biomedical Engineering and Imaging Sciences, King's College London, UK
Centre for Medical Image Computing, University College London, UK
, Lauge SørensenBiomediq A/S, Denmark
Cerebriu A/S, Denmark
University of Copenhagen, Denmark
, Vikram VenkatraghavanBiomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Netherlands, Keli LiuGenentech, USA, Christina RabeGenentech, USA, Paul ManserGenentech, USA, Steven M. HillMRC Biostatistics Unit, University of Cambridge, UK, James HowlettMRC Biostatistics Unit, University of Cambridge, UK, Zhiyue HuangMRC Biostatistics Unit, University of Cambridge, UK, Steven KiddleMRC Biostatistics Unit, University of Cambridge, UK, Sach MukherjeeGerman Center for Neurodegenerative Diseases, Bonn, Germany, Anaïs RouanetMRC Biostatistics Unit, University of Cambridge, UK, Bernd TaschlerGerman Center for Neurodegenerative Diseases, Bonn, Germany, Brian D. M. TomMRC Biostatistics Unit, University of Cambridge, UK, Simon R. WhiteMRC Biostatistics Unit, University of Cambridge, UK, Noel FauxIBM Research Australia, Melbourne, Australia, Suman SedaiIBM Research Australia, Melbourne, Australia, Javier de Velasco OriolInstituto Tecnológico y de Estudios Superiores de Monterrey, Monterrey, Mexico, Edgar E. V. ClementeInstituto Tecnológico y de Estudios Superiores de Monterrey, Monterrey, Mexico, Karol EstradaBrandeis University, Waltham, USA
Department of Statistical Genetics, Biomarin, San Rafael, USA
, Leon AksmanCentre for Medical Image Computing, University College London, UK, Andre AltmannCentre for Medical Image Computing, University College London, UK, Cynthia M. StonningtonMayo Clinic, Scottsdale, AZ, USA, Yalin WangSchool of Computing, Informatics and Decision Systems Engineering, Arizona State University, Tempe, USA, Jianfeng WuSchool of Computing, Informatics and Decision Systems Engineering, Arizona State University, Tempe, USA, Vivek DevadasBanner Alzheimer's Institute, Phoenix, USA, Clementine FourrierInstitut du Cerveau et de la Moelle épinière, Paris, France, Lars Lau RaketH. Lundbeck A/S, Denmark
Clinical Memory Research Unit, Department of Clinical Sciences Malmo, Lund University, Lund, Sweden
, Aristeidis SotirasCenter for Biomedical Image Computing and Analytics, University of Pennsylvania, Guray ErusCenter for Biomedical Image Computing and Analytics, University of Pennsylvania, Jimit DoshiCenter for Biomedical Image Computing and Analytics, University of Pennsylvania, Christos DavatzikosCenter for Biomedical Image Computing and Analytics, University of Pennsylvania, Jacob VogelMcGill University, Montreal, Canada, Andrew DoyleMcGill University, Montreal, Canada, Angela TamMcGill University, Montreal, Canada, Alex Diaz-PapkovichMcGill University, Montreal, Canada, Emmanuel JammehUniversity of Plymouth, UK, Igor KovalInstitut du Cerveau et de la Moelle épinière, Paris, France, Paul MooreMathematical Institute, University of Oxford, UK, Terry J. LyonsMathematical Institute, University of Oxford, UK, John GallacherDepartment of Psychiatry, University of Oxford, UK, Jussi TohkaA.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, Finland, Robert CiszekA.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, Finland, Bruno JedynakPortland State University, Portland, USA, Kruti PandyaPortland State University, Portland, USA, Murat BilgelLaboratory of Behavioral Neuroscience, National Institute on Aging, National Institutes of Health, Baltimore, MD, USA, William EngelsPortland State University, Portland, USA, Joseph ColePortland State University, Portland, USA, Polina GollandComputer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, USA, Stefan KleinBiomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Netherlands, Daniel C. AlexanderCentre for Medical Image Computing, University College London, UK, The EuroPOND Consortium Author not affiliated with any research institution, The Alzheimer's Disease Neuroimaging Initiative Author not affiliated with any research institution
December 2021 issue
Publication date: 2021/12/31
PDF · arXiv · Website · TADPOLE-SHARE · Source code · Video

Abstract

Accurate prediction of progression in subjects at risk of Alzheimer's disease is crucial for enrolling the right subjects in clinical trials. However, a prospective comparison of state-of-the-art algorithms for predicting disease onset and progression is currently lacking. We present the findings of "The Alzheimer's Disease Prediction Of Longitudinal Evolution" (TADPOLE) Challenge, which compared the performance of 92 algorithms from 33 international teams at predicting the future trajectory of 219 individuals at risk of Alzheimer's disease. Challenge participants were required to make a prediction, for each month of a 5-year future time period, of three key outcomes: clinical diagnosis, Alzheimer's Disease Assessment Scale Cognitive Subdomain (ADAS-Cog13), and total volume of the ventricles. The methods used by challenge participants included multivariate linear regression, machine learning methods such as support vector machines and deep neural networks, as well as disease progression models. No single submission was best at predicting all three outcomes. For clinical diagnosis and ventricle volume prediction, the best algorithms strongly outperform simple baselines in predictive ability. However, for ADAS-Cog13 no single submitted prediction method was significantly better than random guesswork. Two ensemble methods based on taking the mean and median over all predictions, obtained top scores on almost all tasks. Better than average performance at diagnosis prediction was generally associated with the additional inclusion of features from cerebrospinal fluid (CSF) samples and diffusion tensor imaging (DTI). On the other hand, better performance at ventricle volume prediction was associated with inclusion of summary statistics, such as the slope or maxima/minima of patient-specific biomarkers. On a limited, cross-sectional subset of the data emulating clinical trials, performance of the best algorithms at predicting clinical diagnosis decreased only slightly (2 percentage points) compared to the full longitudinal dataset. The submission system remains open via the website https://tadpole.grand-challenge.org, while TADPOLE SHARE (https://tadpole-share.github.io/) collates code for submissions. TADPOLE's unique results suggest that current prediction algorithms provide sufficient accuracy to exploit biomarkers related to clinical diagnosis and ventricle volume, for cohort refinement in clinical trials for Alzheimer's disease. However, results call into question the usage of cognitive test scores for patient selection and as a primary endpoint in clinical trials.

Keywords

statistical modelling · machine learning · benchmark · alzheimer's disease prediction

Bibtex @article{melba:2021:019:marinescu, title = "The Alzheimer's Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge: Results after 1 Year Follow-up", authors = "Marinescu, Razvan V. and Oxtoby, Neil P. and Young, Alexandra L. and Bron, Esther E. and Toga, Arthur W. and Weiner, Michael W. and Barkhof, Frederik and Fox, Nick C. and Eshaghi, Arman and Toni, Tina and Salaterski, Marcin and Lunina, Veronika and Ansart, Manon and Durrleman, Stanley and Lu, Pascal and Iddi, Samuel and Li, Dan and Thompson, Wesley K. and Donohue, Michael C. and Nahon, Aviv and Levy, Yarden and Halbersberg, Dan and Cohen, Mariya and Liao, Huiling and Li, Tengfei and Yu, Kaixian and Zhu, Hongtu and Tamez-Peña, José G. and Ismail, Aya and Wood, Timothy and Bravo, Hector Corrada and Nguyen, Minh and Sun, Nanbo and Feng, Jiashi and Yeo, B.T. Thomas and Chen, Gang and Qi, Ke and Chen, Shiyang and Qiu, Deqiang and Buciuman, Ionut and Kelner, Alex and Pop, Raluca and Rimocea, Denisa and Ghazi, Mostafa M. and Nielsen, Mads and Ourselin, Sebastien and Sørensen, Lauge and Venkatraghavan, Vikram and Liu, Keli and Rabe, Christina and Manser, Paul and Hill, Steven M. and Howlett, James and Huang, Zhiyue and Kiddle, Steven and Mukherjee, Sach and Rouanet, Anaïs and Taschler, Bernd and Tom, Brian D. M. and White, Simon R. and Faux, Noel and Sedai, Suman and de Velasco Oriol, Javier and Clemente, Edgar E. V. and Estrada, Karol and Aksman, Leon and Altmann, Andre and Stonnington, Cynthia M. and Wang, Yalin and Wu, Jianfeng and Devadas, Vivek and Fourrier, Clementine and Raket, Lars Lau and Sotiras, Aristeidis and Erus, Guray and Doshi, Jimit and Davatzikos, Christos and Vogel, Jacob and Doyle, Andrew and Tam, Angela and Diaz-Papkovich, Alex and Jammeh, Emmanuel and Koval, Igor and Moore, Paul and Lyons, Terry J. and Gallacher, John and Tohka, Jussi and Ciszek, Robert and Jedynak, Bruno and Pandya, Kruti and Bilgel, Murat and Engels, William and Cole, Joseph and Golland, Polina and Klein, Stefan and Alexander, Daniel C. and , The EuroPOND Consortium and , The Alzheimer's Disease Neuroimaging Initiative", journal = "Machine Learning for Biomedical Imaging", volume = "1", issue = "December 2021 issue", year = "2021" }

2021:019 cover