ENHANCE (ENriching Health data by ANnotations of Crowd and Experts): A case study for skin lesion classification
Ralf Raumanns1,2, Gerard Schouten1,2, Max Joosten2, Josien P. W. Pluim2, Veronika Cheplygina3
1: Fontys University of Applied Scienc, 2: Eindhoven University of Technology, 3: IT University of Copenhagen
Publication date: 2021/12/31
https://doi.org/10.59275/j.melba.2021-geb9
Abstract
We present ENHANCE, an open dataset with multiple annotations to complement the existing ISIC and PH2 skin lesion classification datasets. This dataset contains annotations of visual ABC (asymmetry, border, colour) features from non-expert annotation sources: undergraduate students, crowd workers from Amazon MTurk and classic image processing algorithms. In this paper we first analyse the correlations between the annotations and the diagnostic label of the lesion, as well as study the agreement between different annotation sources. Overall we find weak correlations of non-expert annotations with the diagnostic label, and low agreement between different annotation sources. We then study multi-task learning (MTL) with the annotations as additional labels, and show that non-expert annotations can improve (ensembles of) state-of-the-art convolutional neural networks via MTL. We hope that our dataset can be used in further research into multiple annotations and/or MTL. All data and models are available on Github: https://github.com/raumannsr/ENHANCE.
Keywords
Open data · Crowdsourcing · Multi-task learning · Skin cancer · Ensembles · Overfitting
Bibtex
@article{melba:2021:020:raumanns,
title = "ENHANCE (ENriching Health data by ANnotations of Crowd and Experts): A case study for skin lesion classification",
author = "Raumanns, Ralf and Schouten, Gerard and Joosten, Max and Pluim, Josien P. W. and Cheplygina, Veronika",
journal = "Machine Learning for Biomedical Imaging",
volume = "1",
issue = "December 2021 issue",
year = "2021",
pages = "1--26",
issn = "2766-905X",
doi = "https://doi.org/10.59275/j.melba.2021-geb9",
url = "https://melba-journal.org/2021:020"
}
RIS
TY - JOUR
AU - Raumanns, Ralf
AU - Schouten, Gerard
AU - Joosten, Max
AU - Pluim, Josien P. W.
AU - Cheplygina, Veronika
PY - 2021
TI - ENHANCE (ENriching Health data by ANnotations of Crowd and Experts): A case study for skin lesion classification
T2 - Machine Learning for Biomedical Imaging
VL - 1
IS - December 2021 issue
SP - 1
EP - 26
SN - 2766-905X
DO - https://doi.org/10.59275/j.melba.2021-geb9
UR - https://melba-journal.org/2021:020
ER -