Purpose RobotReviewer is a machine learning system for semi‐automated assistance in risk of bias assessment. The tools’s performance in randomized controlled trials (RCTs) in the field of nursing remains unknown. We aimed therefore to evaluate the agreement in risk of bias assessment between RobotReviewer and human reviewers. Design Evaluation study using a retrospective diagnostic design. Methods We used RobotReviewer as the index test and human reviewers’ risk of bias assessment reported in Cochrane reviews as the reference test. A convenience sample of electronically available English‐language full texts of RCTs included in Cochrane reviews with nurs* in the title were eligible for inclusion. In this context, we assessed random sequence generation, allocation concealment, and blinding (personnel or participants and assessors) corresponding to Cochrane risk of bias version 2011. Two independent research teams performed and double‐checked data extraction and analysis. We calculated sensitivity, specificity, receiver operating characteristic (ROC) curve, the area under the ROC curve, predictive values, observed percentage of agreement, and Cohen’s kappa (including confidence intervals, if applicable). Findings The selection process yielded 190 RCTs published between 1958 and 2016 in 23 Cochrane reviews published between 2000 and 2018. Missing assessments of risk of bias domains in Cochrane reviews or RobotReviewer yielded varying sample sizes per risk of bias domain. Sensitivity ranged from 0.44 to 0.88 and specificity from 0.48 to 0.95. Positive predictive value was highest for allocation concealment (0.79) and lowest for blinding assessors (0.25). Cohen’s kappa was moderate for randomization (0.52), allocation concealment (0.60), and for blinding of personnel/patients (0.43). Blinding of outcome assessors had only slight agreement (0.04). Conclusions This is the first evaluation of risk of bias assessment by RobotReviewer in RCTs included in nursing‐related Cochrane reviews. It yielded a moderate degree of agreement with human reviewers for randomization and allocation concealment, and an adequate sensitivity for detecting low risk of selection bias. Clinical Relevance Based on our results, using the RobotReviewer for risk of bias assessment in RCTs can be supportive in some risk of bias domains. However, human reviewers should supervise the semi‐automated assessment process.
Objective: The authors reviewed educational interventions for improving literature searching skills in the health sciences.Methods: We performed a scoping review of experimental and quasi-experimental studies published in English and German, irrespective of publication year. Targeted outcomes were objectively measurable literature searching skills (e.g., quality of search strategy, study retrieval, precision). The search methods consisted of searching databases (CINAHL, Embase, MEDLINE, PsycINFO, Web of Science), tracking citations, free web searching, and contacting experts. Two reviewers performed screening and data extraction. To evaluate the completeness of reporting, the Template for Intervention Description and Replication (TIDieR) was applied.Results: We included 6 controlled trials and 8 pre-post trials from the 8,484 references that we screened. Study participants were students in various health professions and physicians. The educational formats of the interventions varied. Outcomes clustered into 2 categories: (1) developing search strategies (e.g., identifying search concepts, selecting databases, applying Boolean operators) and (2) database searching skills (e.g., searching PubMed, MEDLINE, or CINAHL). In addition to baseline and post-intervention measurement, 5 studies reported follow-up. Almost all studies adequately described their intervention procedures and delivery but did not provide access to the educational material. The expertise of the intervention facilitators was described in only 3 studies.Conclusions: The results showed a wide range of study populations, interventions, and outcomes. Studies often lacked information about educational material and facilitators’ expertise. Further research should focus on intervention effectiveness using controlled study designs and long-term follow-up. To ensure transparency, replication, and comparability, studies should rigorously describe their intervention. This article has been approved for the Medical Library Association’s Independent Reading Program.
Background and PurposeThe Reconceptualized Uncertainty in Illness Theory (RUIT) includes the concept of “probabilistic thinking” intending to explain the positive reappraisal of uncertainty in chronic illness. However, the description of the concept is vague, thereby limiting the understanding of the theory. Thus, the aim was to develop a theoretical definition of probabilistic thinking in order to increase the explanatory value of RUIT.MethodsWe conducted a principle-based concept analysis by means of a conceptually driven literature search. Methods consisted of database, dictionary, lexicon, and free web searching as well as citation tracking. We analyzed the concept in terms of (a) epistemology, (b) pragmatics, (c) logic, and (d) linguistics.ResultsThe final data set included 27 publications, 14 of them from nursing. (a) Probabilistic thinking is a coping strategy to handle uncertainty. It involves a focus on either possibilities (in nursing) or probabilities (in other disciplines). (b) There is a lack of operationalization in nursing, though three measurements focusing the handling of probabilities are offered in psychology. (c) Nursing authors interpreting probabilistic thinking as accepted uncertainty lacked logical appropriateness, since probability negotiates uncertainty. (d) Probabilistic thinking is used synonymously with positive thinking and probabilistic reasoning.Implications for PracticeNurses working with chronically ill patients should consider the findings for the application of RUIT. They should recognize whether uncertainty is perceived as a danger and encourage probabilistic thinking. Efforts are necessary to achieve a common language between nursing and other disciplines in order to avoid misunderstandings in clinical practice and research.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.