Machine learning for health must be reproducible to ensure reliable clinical use. We evaluated 511 scientific papers across several machine learning subfields and found that machine learning for health compared poorly to other areas regarding reproducibility metrics, such as dataset and code accessibility. We propose recommendations to address this problem.

No abstract

<b><i>Introduction:</i></b> A major challenge in the monitoring of rehabilitation is the lack of long-term individual baseline data which would enable accurate and objective assessment of functional recovery. Consumer-grade wearable devices enable the tracking of individual everyday functioning prior to illness or other medical events which necessitate the monitoring of recovery trajectories. <b><i>Methods:</i></b> For 1,324 individuals who underwent surgery on a lower limb, we collected their Fitbit device data of steps, heart rate, and sleep from 26 weeks before to 26 weeks after the self-reported surgery date. We identified subgroups of individuals who self-reported surgeries for bone fracture repair (<i>n</i> = 355), tendon or ligament repair/reconstruction (<i>n</i> = 773), and knee or hip joint replacement (<i>n</i> = 196). We used linear mixed models to estimate the average effect of time relative to surgery on daily activity measurements while adjusting for gender, age, and the participant-specific activity baseline. We used a sub-cohort of 127 individuals with dense wearable data who underwent tendon/ligament surgery and employed XGBoost to predict the self-reported recovery time. <b><i>Results:</i></b> The 1,324 study individuals were all US residents, predominantly female (84%), white or Caucasian (85%), and young to middle-aged (mean age 36.2 years). We showed that 12 weeks pre- and 26 weeks post-surgery trajectories of daily behavioral measurements (steps sum, heart rate, sleep efficiency score) can capture activity changes relative to an individual’s baseline. We demonstrated that the trajectories differ across surgery types, recapitulate the documented effect of age on functional recovery, and highlight differences in relative activity change across self-reported recovery time groups. Finally, using a sub-cohort of 127 individuals, we showed that long-term recovery can be accurately predicted, on an individual level, only 1 month after surgery (AUROC 0.734, AUPRC 0.8). Furthermore, we showed that predictions are most accurate when long-term, individual baseline data are available. <b><i>Discussion:</i></b> Leveraging long-term, passively collected wearable data promises to enable relative assessment of individual recovery and is a first step towards data-driven intervention for individuals.

We conducted a survey about recent surgical procedures on a large connected population and requested each individual's permission to access data from commercial wearable devices they may have been wearing around the time of the procedure. For subcohorts of 66-118 patients who reported having a weight loss procedure and who had dense Fitbit data around their procedure date, we examined several daily measures of behavior and physiology in the 12 weeks leading up to and the 12 weeks following their procedures. We found that the weeks following weight loss operations were associated with fewer daily total steps, smaller proportions of the day spent walking, lower resting and 95th percentile heart rates, more total sleep time, and greater sleep efficiency. We demonstrate that consumer-grade activity trackers can capture behavioral and physiological changes resulting from weight loss surgery and these devices have the potential to be used to develop measures of patients' postoperative recovery that are convenient, sensitive, scalable, individualized, and continuous.

Background: Since the beginning of the COVID-19 pandemic data from smartphones and connected sensors has been used to learn about symptoms presentation and management outside the clinic walls. However, reports on the validity of such data are still sparse, especially when it comes to symptom progression and relevance of wearable sensors. Objective: To understand the relevance of Person-GeneratedHealth Data (PGHD) as a means for early detection, monitoring and management of COVID-19 in everyday life. This includes quantifying prevalence and progression of symptoms from self-reports as well as changes in activity and physiological parameters continuously measured from wearable sensors, and contextualizing findings for COVID-19 patients with those from cohorts of flu patients. Design, Setting, and Participants: Retrospective digital cohort study of individuals with a self-reported positive SARS-CoV-2 or influenza test followed over the period 2019-12-02 to2020-04-27. Three cohorts were derived: Patients who self-reported being diagnosed with flu prior to the SARS-CoV-2 pandemic (N=6270, of which 1226 also contributed sensorPGHD); Patients who reported being diagnosed with flu during the SARS-CoV-2 pandemic (N=426, of which 85 also shared sensor PGHD); and patients who reported being diagnosed withCOVID-19 (N=230, of which sensor PGHD was available for 41).The cohorts were derived from a large-scale digital participatory surveillance study designed to track Influenza-like Illness(ILI) incidence and burden over time. Exposures: Self-reported demographic data, comorbidities, and symptoms experienced during a diagnosed ILI episode, including SARS-CoV-2.Physiological and behavioral parameters measured daily from commercial wearable sensors, includingResting Heart Rate (RHR), total step count, and nightly sleep hours. Main Outcomes and Measures: We investigated the percent-age of individuals experiencing symptoms of a given type (e.g.shortness of breath) across demographic groups and over time. We examined illness duration, and care seeking behavior, and how RHR, step count, and nightly sleep hours deviated from expected behavior on healthy days over the course of the infection episode. Results: Self-reported symptoms of COVID-19 present differently from flu. COVID-19 cases tended to last longer than flu(median of 12 vs. 9 days), are uniquely characterized by chest pain/pressure, shortness of breath, and anosmia. The fraction of elevated RHR measurements collected daily from commercial wearable devices rise significantly in the 2 days surrounding ILI symptoms onset, but does not appear to do so in a way specific to COVID-19. Steps lost due to COVID-19 persists for longer. Conclusion and Relevance: PGHD can be a valid source of longitudinal real world data to detect and monitor COVID-19-related symptoms and behaviors at population scale. PGHD may provide continuous, near realtime feedback to intervention effectiveness that otherwise requires waiting for symptoms to develop into contacts with the healthcare system. It has also the potential to increase pre-test probability of other downstream diagnostics. To effectively leverage PGHD for participatory surveillance it is crucial to invest in the creation of trusted, long-term communication channels with individuals through whichdata can be efficiently collected, consented, and contextualized,while protecting the privacy of individuals and ultimately facilitating the transition in and out of care.