Human Computation as a New Method for Evidence-Based Knowledge Transfer in Web-Based Guideline Development Groups: Proof of Concept Randomized Controlled Trial

Abstract

BackgroundGuideline developers use different consensus methods to develop evidence-based clinical practice guidelines. Previous research suggests that existing guideline development techniques are subject to methodological problems and are logistically demanding. Guideline developers welcome new methods that facilitate a methodologically sound decision-making process. Systems that aggregate knowledge while participants play a game are one class of human computation applications. Researchers have already proven that these games with a purpose are effective in building common sense knowledge databases.ObjectiveWe aimed to evaluate the feasibility of a new consensus method based on human computation techniques compared to an informal face-to-face consensus method.MethodsWe set up a randomized design to study 2 different methods for guideline development within a group of advanced students completing a master of nursing and obstetrics. Students who participated in the trial were enrolled in an evidence-based health care course. We compared the Web-based method of human-based computation (HC) with an informal face-to-face consensus method (IC). We used 4 clinical scenarios of lower back pain as the subject of the consensus process. These scenarios concerned the following topics: (1) medical imaging, (2) therapeutic options, (3) drugs use, and (4) sick leave. Outcomes were expressed as the amount of group (dis)agreement and the concordance of answers with clinical evidence. We estimated within-group and between-group effect sizes by calculating Cohen’s d. We calculated within-group effect sizes as the absolute difference between the outcome value at round 3 and the baseline outcome value, divided by the pooled standard deviation. We calculated between-group effect sizes as the absolute difference between the mean change in outcome value across rounds in HC and the mean change in outcome value across rounds in IC, divided by the pooled standard deviation. We analyzed statistical significance of within-group changes between round 1 and round 3 using the Wilcoxon signed rank test. We assessed the differences between the HC and IC groups using Mann-Whitney U tests. We used a Bonferroni adjusted alpha level of .025 in all statistical tests. We performed a thematic analysis to explore participants’ arguments during group discussion. Participants completed a satisfaction survey at the end of the consensus process.ResultsOf the 135 students completing a master of nursing and obstetrics, 120 participated in the experiment. We formed 8 HC groups (n=64) and 7 IC groups (n=56). The between-group comparison demonstrated that the human computation groups obtained a greater improvement in evidence scores compared to the IC groups, although the difference was not statistically significant. The between-group effect size was 0.56 (P=.30) for the medical imaging scenario, 0.07 (P=.97) for the therapeutic options scenario, and 0.89 (P=.11) for the drug use scenario. We found no significant differences in improvement in the degree of agree...

Escape room as an adult education tool

“…Since the 1940s, when Reginald Revans (1980) developed formal assumptions and laid the foundations for the so-called action learning, games paved their way as full-fledged education tools through games and simulations used in business schools and in professional development of large organisations employees, up to current numerous examples of traditional and electronic games used in schools and universities. It is difficult to unequivocally determine a moment in time when games were accepted by practitioners; but with their dissemination ever more terms describing games used in nonentertainment contexts surfaced: serious games (Kleitmann, Narciss, 2019), edutainment (Rattakom, Sanit, Ladawan, 2019), human-based computation games (Heselmans et al, 2013) or games with a purpose (Amiri-Chimeh et al, 2018). Games have found their place and stayed for good in training and recruitment (Raybourn, 2007), personal development (Kai et al, 2020), marketing (Arora, Saxena, 2015), or risk analysis (Wang, Neil, Fenton, 2020).…”

Use of m-Health Technology for Preventive Interventions to Tackle Cardiometabolic Conditions and Other Non-Communicable Diseases in Latin America- Challenges and Opportunities

“…34,35 Interventions that have been effective in dealing with barriers related to clinical practice are educational interventions (academic detailing, educational outreach visits, audit and feed-back) and clinical decision support systems (CDSS). [36][37][38] Mobile phones are recognized as a viable platform for improving health care delivery in lowresource settings in developed countries. 20,39 With the rapid uptake and use of smartphone technology, there has been a surge in the development of mobile applications (apps) for health.…”