TR&D1 Advances Missing Data Imputation for Wearable Biomarkers with Transformer Models
TR&D1 has achieved a major advancement in missing data imputation for wearable health biomarkers, particularly focusing on hourly step count data. Handling missing or irregularly recorded data is a persistent challenge in mHealth studies, and accurate imputation is critical for downstream analyses and intervention design. Building on work from the HeartSteps II trial (CP3), the team has now expanded their approach to over 5.5 million observations from 10,000 participants in the All of Us Research Project (SP1).
The team developed a custom transformer model incorporating a domain-knowledge-informed, multi-timescale self-attention mechanism. This design allows the model to capture patterns at varying temporal resolutions while handling irregularities and rare, outlying events in step count data. The approach has demonstrated superior performance over existing imputation techniques, substantially improving the accuracy and reliability of reconstructed biomarker data.
These findings were presented at the International Conference on Learning Representations workshop on Learning from Time Series for Health (ICLR 2024) and as a full paper at the Conference on Health, Inference, and Learning (CHIL 2024). The work represents a critical methodological advancement, enabling more precise modeling of wearable sensor data and supporting robust analyses of large-scale mHealth datasets.
By leveraging transformer-based architectures tailored to health data, TR&D1 is providing a scalable and accurate solution for missing data imputation, ensuring researchers and clinicians can rely on wearable-derived biomarkers to guide interventions and inform health insights.
Study citation
Wei, H., Xu, M. A., Samplawski, C., Rehg, J. M., Kumar, S., & Marlin, B. M. (2024). Temporally multi-scale sparse self-attention for physical activity data imputation. In Proceedings of the Conference on Health, Inference, and Learning (CHIL 2024). https://arxiv.org/abs/2406.18848