Population-aware hierarchical bayesian domain adaptation via multi-component invariant learning

Abstract

While machine learning is rapidly being developed and deployed in health settings such as influenza prediction, there are critical challenges in using data from one environment in another due to variability in features; even within disease labels there can be differences (e.g. "fever" may mean something different reported in a doctor's office versus in an online app). Moreover, models are often built on passive, observational data which contain different distributions of population subgroups (e.g. men or women). Thus, there are two forms of instability between environments in this observational transport problem. We first harness knowledge from health to conceptualize the underlying causal structure of this problem in a health outcome prediction task. Based on sources of stability in the model, we posit that for human-sourced data and health prediction tasks we can combine environment and population information in a novel population-aware hierarchical Bayesian domain adaptation framework that harnesses multiple invariant components through population attributes when needed. We study the conditions under which invariant learning fails, leading to reliance on the environment-specific attributes. Experimental results for an influenza prediction task on four datasets gathered from different contexts show the model can improve prediction in the case of largely unlabelled target data from a new environment and different constituent population, by harnessing both environment and population invariant information. This work represents a novel, principled way to address a critical challenge by blending domain (health) knowledge and algorithmic innovation. The proposed approach will have significant impact in many social settings wherein who and where the data comes from matters.

Machine learning and algorithmic fairness in public and population health

“…The often-linked geographic information and time stamps further enable the capture of hyper-local, daily and sub-daily health-related information from behaviours to exposures and other macro-level properties, as well as health outcomes 23,24 . Accordingly, such ubiquitous technologies can provide opportunities to better measure the social determinants of health in a targeted way, by person, location and/or time [21][22][23][24][25] . These attributes of person-generated data can complement denominator-based survey and report data that are not available at such high granularity.…”

“…Domain adaptation methods can be developed to address distribution shifts that may occur across different environment, both based on different populations and data generation mechanisms. 25,63,64 In particular, thinking about data distribution shifts and differences from a causal perspective has been utilized to inform the empirical learning processes 23,65 .…”

Bayesian Nested Latent Class Models for Cause-of-Death Assignment using Verbal Autopsies Across Multiple Domains

“…The domain adaptation problem refers to the situation where a statistical learning model trained on one labeled dataset needs to be generalized to the target dataset, or target domain, drawn from a different distribution and with insufficient labeled data (Daume III and Marcu, 2006). Learning from data collected in different domains is an active area of research in computer science and has been explored in various applications including natural language processing (Ramponi andPlank, 2020), visual classification (Wang andDeng, 2018), sentiment prediction (Glorot et al, 2011), and more recently in prediction problems in public health and clinical settings (Rehman et al, 2018;Mhasawade et al, 2020;Laparra et al, 2020).…”

Machine Learning in Population and Public Health

“…For example, recent work on how individual-level syndromic reports or passive data from individuals relates to microbiologic confirmation [14], aims to address early challenges highlighted in using data such as Google searches to predict influenza [29]. Research on the data generating process of new digital data sources is also imperative to understand what the data represents, and why data is being shared by individuals [37]. Second, can the results be extended to other situations, groups, or events; which sub-populations are representative of the hypothesis [38]?…”

“…Even if similar syndromic data is collected across all the approaches, there are considerable differences in the predictive value of the syndromic data [12], highlighting that the same type of data can represent different factors across studies (in this case the mode of data collection can affect the specificity/sensitivity of respiratory infection syndromic data). Although there are significant differences in sample representation and predictivity across studies, it is crucial to understand external validity as it provides a way to understand populationlevel characteristics that remain invariant across studies [11,37].…”