BackgroundThere is growing interest in using machine learning approaches to priority rank studies and reduce human burden in screening literature when conducting systematic reviews. In addition, identifying addressable questions during the problem formulation phase of systematic review can be challenging, especially for topics having a large literature base. Here, we assess the performance of the SWIFT-Review priority ranking algorithm for identifying studies relevant to a given research question. We also explore the use of SWIFT-Review during problem formulation to identify, categorize, and visualize research areas that are data rich/data poor within a large literature corpus.MethodsTwenty case studies, including 15 public data sets, representing a range of complexity and size, were used to assess the priority ranking performance of SWIFT-Review. For each study, seed sets of manually annotated included and excluded titles and abstracts were used for machine training. The remaining references were then ranked for relevance using an algorithm that considers term frequency and latent Dirichlet allocation (LDA) topic modeling. This ranking was evaluated with respect to (1) the number of studies screened in order to identify 95 % of known relevant studies and (2) the “Work Saved over Sampling” (WSS) performance metric. To assess SWIFT-Review for use in problem formulation, PubMed literature search results for 171 chemicals implicated as EDCs were uploaded into SWIFT-Review (264,588 studies) and categorized based on evidence stream and health outcome. Patterns of search results were surveyed and visualized using a variety of interactive graphics.ResultsCompared with the reported performance of other tools using the same datasets, the SWIFT-Review ranking procedure obtained the highest scores on 11 out of 15 of the public datasets. Overall, these results suggest that using machine learning to triage documents for screening has the potential to save, on average, more than 50 % of the screening effort ordinarily required when using un-ordered document lists. In addition, the tagging and annotation capabilities of SWIFT-Review can be useful during the activities of scoping and problem formulation.ConclusionsText-mining and machine learning software such as SWIFT-Review can be valuable tools to reduce the human screening burden and assist in problem formulation.Electronic supplementary materialThe online version of this article (doi:10.1186/s13643-016-0263-z) contains supplementary material, which is available to authorized users.

No abstract

Background The objective of this evaluation is to understand the human health impacts of mountaintop removal (MTR) mining, the major method of coal mining in and around Central Appalachia. MTR mining impacts the air, water, and soil and raises concerns about potential adverse health effects in neighboring communities;exposures associated with MTR mining include particulate matter (PM), polycyclic aromatic hydrocarbons (PAHs), metals, hydrogen sulfide, and other recognized harmful substances. Methods A systematic review was conducted of published studies of MTR mining and community health, occupational studies of MTR mining, and any available animal and in vitro experimental studies investigating the effects of exposures to MTR-mining-related chemical mixtures. Six databases (Embase, PsycINFO, PubMed, Scopus, Toxline, and Web of Science) were searched with customized terms, and no restrictions on publication year or language, through October 27, 2016. The eligibility criteria included all human population studies and animal models of human health, direct and indirect measures of MTR-mining exposure, any health-related effect or change in physiological response, and any study design type. Risk of bias was assessed for observational and experimental studies using an approach developed by the National Toxicology Program (NTP) Office of Health Assessment and Translation (OHAT). To provide context for these health effects, a summary of the exposure literature is included that focuses on describing findings for outdoor air, indoor air, and drinking water. Results From a literature search capturing 3088 studies, 33 human studies (29 community, four occupational), four experimental studies (two in rat, one in vitro and in mice, one in C. elegans), and 58 MTR mining exposure studies were identified. A number of health findings were reported in observational human studies, including cardiopulmonary effects, mortality, and birth defects. However, concerns for risk of bias were identified, especially with respect to exposure characterization, accounting for confounding variables (such as socioeconomic status), and methods used to assess health outcomes. Typically, exposure was assessed by proximity of residence or hospital to coal mining or production level at the county level. In addition, assessing the consistency of findings was challenging because separate publications likely included overlapping case and comparison groups. For example, 11 studies of mortality were conducted with most reporting higher rates associated with coal mining, but many of these relied on the same national datasets and were unable to consider individual-level contributors to mortality such as poor socioeconomic status or smoking. Two studies of adult rats reported impaired microvascular and cardiac mitochondrial function after intratracheal exposure to PM from MTR-mining sites. Exposures associated with MTR mining included reports of PM levels that sometimes exceeded Environmental Protection Agency (EPA) standards; higher levels of dust, tra...