A Data-Driven Correction Method for Participant-Reported Labels in Wearable Health Datasets

Abstract

The widespread use and advanced capabilities of modern Commercial Off-The-Shelf (COTS) wearables provide the unprecedented opportunity for rapid detection and early notification of adverse physiological events. Data-driven models derived from COTS wearable datasets often rely on participant-reported (i.e., non-expert) labels for detection of these events. These labels, however, suffer from poor user compliance leading to inaccurate event representations. We propose a novel correction algorithm, Probability Voting, which utilizes the information and temporal relationships between data samples to correct labels prior to model training. We assessed this method on potentially one of the largest COTS wearables datasets, comprised of over 8000 participants and +1.3m hours of collected data. We show improvements of ROC AUCs by 6.1% for the prediction of COVID-19, 7.2% for the detection of shortness of breath, 1.4% for the detection of fever, and 2.8% for the detection of elevated temperature. Across all selected metrics, including ROC AUC, PR AUC, sensitivity at 75% specificity, and precision at 75% recall, all metrics improved over baseline using the proposed label correction method on the training set. We also analyze the effects of this preprocessing technique on the structure of training data after applying label correction and demonstrate its reliance on physiologically related features despite its semi-supervised approach.