Naive Bayes classifiers for verbal autopsies: comparison to physician-based classification for 21,000 child and adult deaths

Abstract

BackgroundVerbal autopsies (VA) are increasingly used in low- and middle-income countries where most causes of death (COD) occur at home without medical attention, and home deaths differ substantially from hospital deaths. Hence, there is no plausible “standard” against which VAs for home deaths may be validated. Previous studies have shown contradictory performance of automated methods compared to physician-based classification of CODs. We sought to compare the performance of the classic naive Bayes classifier (NBC) versus existing automated classifiers, using physician-based classification as the reference.MethodsWe compared the performance of NBC, an open-source Tariff Method (OTM), and InterVA-4 on three datasets covering about 21,000 child and adult deaths: the ongoing Million Death Study in India, and health and demographic surveillance sites in Agincourt, South Africa and Matlab, Bangladesh. We applied several training and testing splits of the data to quantify the sensitivity and specificity compared to physician coding for individual CODs and to test the cause-specific mortality fractions at the population level.ResultsThe NBC achieved comparable sensitivity (median 0.51, range 0.48-0.58) to OTM (median 0.50, range 0.41-0.51), with InterVA-4 having lower sensitivity (median 0.43, range 0.36-0.47) in all three datasets, across all CODs. Consistency of CODs was comparable for NBC and InterVA-4 but lower for OTM. NBC and OTM achieved better performance when using a local rather than a non-local training dataset. At the population level, NBC scored the highest cause-specific mortality fraction accuracy across the datasets (median 0.88, range 0.87-0.93), followed by InterVA-4 (median 0.66, range 0.62-0.73) and OTM (median 0.57, range 0.42-0.58).ConclusionsNBC outperforms current similar COD classifiers at the population level. Nevertheless, no current automated classifier adequately replicates physician classification for individual CODs. There is a need for further research on automated classifiers using local training and test data in diverse settings prior to recommending any replacement of physician-based classification of verbal autopsies.Electronic supplementary materialThe online version of this article (doi:10.1186/s12916-015-0521-2) contains supplementary material, which is available to authorized users.

Measuring mortality due to HIV-associated tuberculosis among adults in South Africa: Comparing verbal autopsy, minimally-invasive autopsy, and research data

“…tuberculosis , which likely led to the exclusion of individuals with disseminated TB and limited or no respiratory symptoms. To date, all comparisons of VA to the PHMRC dataset, including those conducted by the PHMRC team, have combined the ‘AIDS’ and ‘AIDS with TB’ categories, and have therefore not attempted to assess VA’s ability to detect HIV-associated TB [19,20,30,60–64]. The PHMRC gold standard dataset nevertheless remains a valuable resource; we would suggest that any future validation exercises use the differentiated, ‘AIDS with TB’ and ‘AIDS’ categories, rather than the combined ‘AIDS’ category, for comparison to VA.…”

Regularized Bayesian transfer learning for population-level etiological distributions

“…The openVA R-package (Li et al, 2018) has made many of these algorithms publicly available. Generic classifiers like random forests (Breiman, 2001), naive Bayes classifiers (Minsky, 1961) and support vector machines (Cortes and Vapnik, 1995) have also been used Miasnikof et al, 2015;Koopman et al, 2015). Estimated COD labels for each VA record in a nationally representative VA database is aggregated to obtain national cause specific mortality fractions (CSMF) -the population-level class membership probabilities, that are often the main quantities of interest for epidemiologists, local governments, and global health organizations.…”

Using Bayesian Latent Gaussian Graphical Models to Infer Symptom Associations in Verbal Autopsies

“…This extension allows us to complete our model to simultaneously estimate the latent correlation matrix and assign causes of death using VA data. Before we describe our model, it is worth noting that for many existing automated VA methods such as InSilicoVA (McCormick et al, 2016), InterVA (Byass et al, 2003), and the Naive Bayes Classifier (Miasnikof et al, 2015), the classification rule is closely related to the naive Bayes classifier under the assumption of (conditional) independence between symptoms, i.e.…”

“…The majority of the existing statistical or algorithmic methods to assign cause of death using VA surveys make the assumption that VA symptoms are independent from one another conditional on cause of death (Byass et al, 2003;James et al, 2011;Miasnikof et al, 2015;McCormick et al, 2016). This assumption simplifies computation and is efficient in settings with limited training data.…”