Performance of four computer-coded verbal autopsy methods for cause of death assignment compared with physician coding on 24,000 deaths in low- and middle-income countries

Abstract

BackgroundPhysician-coded verbal autopsy (PCVA) is the most widely used method to determine causes of death (CODs) in countries where medical certification of death is uncommon. Computer-coded verbal autopsy (CCVA) methods have been proposed as a faster and cheaper alternative to PCVA, though they have not been widely compared to PCVA or to each other.MethodsWe compared the performance of open-source random forest, open-source tariff method, InterVA-4, and the King-Lu method to PCVA on five datasets comprising over 24,000 verbal autopsies from low- and middle-income countries. Metrics to assess performance were positive predictive value and partial chance-corrected concordance at the individual level, and cause-specific mortality fraction accuracy and cause-specific mortality fraction error at the population level.ResultsThe positive predictive value for the most probable COD predicted by the four CCVA methods averaged about 43% to 44% across the datasets. The average positive predictive value improved for the top three most probable CODs, with greater improvements for open-source random forest (69%) and open-source tariff method (68%) than for InterVA-4 (62%). The average partial chance-corrected concordance for the most probable COD predicted by the open-source random forest, open-source tariff method and InterVA-4 were 41%, 40% and 41%, respectively, with better results for the top three most probable CODs. Performance generally improved with larger datasets. At the population level, the King-Lu method had the highest average cause-specific mortality fraction accuracy across all five datasets (91%), followed by InterVA-4 (72% across three datasets), open-source random forest (71%) and open-source tariff method (54%).ConclusionsOn an individual level, no single method was able to replicate the physician assignment of COD more than about half the time. At the population level, the King-Lu method was the best method to estimate cause-specific mortality fractions, though it does not assign individual CODs. Future testing should focus on combining different computer-coded verbal autopsy tools, paired with PCVA strengths. This includes using open-source tools applied to larger and varied datasets (especially those including a random sample of deaths drawn from the population), so as to establish the performance for age- and sex-specific CODs.

Measuring mortality due to HIV-associated tuberculosis among adults in South Africa: Comparing verbal autopsy, minimally-invasive autopsy, and research data

“…tuberculosis , which likely led to the exclusion of individuals with disseminated TB and limited or no respiratory symptoms. To date, all comparisons of VA to the PHMRC dataset, including those conducted by the PHMRC team, have combined the ‘AIDS’ and ‘AIDS with TB’ categories, and have therefore not attempted to assess VA’s ability to detect HIV-associated TB [19,20,30,60–64]. The PHMRC gold standard dataset nevertheless remains a valuable resource; we would suggest that any future validation exercises use the differentiated, ‘AIDS with TB’ and ‘AIDS’ categories, rather than the combined ‘AIDS’ category, for comparison to VA.…”

“…In areas with poor civil registration systems, VA data are used to generate estimates of national and regional mortality [18] which may then be used to influence health policy. The last 10 years have seen the development of several automated methods (computer-coded VA [CCVA]) which, it is hoped, will eventually replace the expensive and time-consuming physician-certified VA (PCVA) [19,20]. Numerous studies have attempted to validate VA, but the vast majority compare VA CoD to clinical CoD derived from physician review of hospital or clinic records, a gold standard of variable quality and consistency [21–24].…”

New challenges for verbal autopsy: Considering the ethical and social implications of verbal autopsy methods in routine health information systems

“…COD have been developed (Desai et al, 2014,Murray et al, 2014. VA questionnaires can be analysed by physicians and/or by automated analytical software packages (the accuracy of VA described above was based on an analysis of automated methods).…”

Renal failure deaths and their risk factors in India 2001–13: nationally representative estimates from the Million Death Study

“…Details of the MDS, including study design, physician assignment of the underlying cause of death, and statistical methods have been published elsewhere. [10][11][12] Briefl y, the study uses an enhanced type of verbal autopsy method (a structured survey administered to a household member or close relative of the deceased by a trained non-medical fi eld worker to record the signs and symptoms that occurred before death-this information is used to assign the most probable cause of death 13 ). From 2001 onwards, the MDS has monitored annual deaths in 1·3 million representative households within the Registrar General of India (RGI)'s Sample Registration System (SRS).…”