Learners' progress within computer-supported collaborative learning environments is typically measured via analysis and interpretation of quantitative web interaction measures. However, the usefulness of these "proxies for learning" is questioned as they do not necessarily reflect critical thinking-an essential component of collaborative learning. Research indicates that pedagogical content analysis schemes have value in measuring critical discourse in small scale, formal, online learning environments, but research using these methods on high volume, informal, Massive Open Online Course (MOOC) forums is less common. The challenge in this setting is to develop valid and reliable indicators that operate successfully at scale. In this study, we test two established coding schemes used for the pedagogical content analysis of online discussions in a large-scale review of MOOC comment data. Pedagogical Scores are derived from manual ratings applied to comments by raters and correlated with automatically derived linguistic and interaction indicators. Results show that the content analysis methods are reliable, and are very strongly correlated with each other, suggesting that their specific format is not significant in this setting. In addition, the methods are strongly associated with the relevant linguistic indicators of higher levels of learning and have weaker correlations with other linguistic and interaction metrics. This suggests promise for further research using Machine Learning techniques, with the goal of providing realistic feedback to instructors, learners, and learning designers.
Objectives: To test whether a newly developed person-, theory- and evidence-based website about acupuncture helps patients make informed decisions about whether or not to use acupuncture for back pain. Methods: A randomised online study compared a newly developed ‘enhanced website’ to a ‘standard website’. The enhanced website provided evidence-based information in a person-based manner and targeted psychological constructs. The standard website was based on a widely used patient information leaflet. In total, 350 adults with recent self-reported back pain were recruited from general practices in South West England. The two primary outcomes were knowledge change and making an informed choice about using acupuncture. Secondary outcomes were beliefs about and willingness to have acupuncture. Results: Participants who viewed the enhanced acupuncture website had a significantly greater increase in knowledge about acupuncture (M = 1.1, standard deviation (SD) = 1.7) than participants who viewed the standard website (M = 0.2, SD = 1.1; F(1, 315) = 37.93, p < 0.001, η2 = .107). Participants who viewed the enhanced acupuncture website were also 3.3 times more likely to make an informed choice about using acupuncture than those who viewed the standard website (χ2(1) = 23.46, p < 0.001). There were no significant effects on treatment beliefs or willingness to have acupuncture. Conclusion: The enhanced website improved patients’ knowledge and ability to make an informed choice about acupuncture, but did not optimise treatment beliefs or change willingness to have acupuncture. The enhanced website could be used to support informed decision-making among primary care patients and members of the general public considering using acupuncture for back pain.
IntroductionPotential acupuncture patients seek out information about acupuncture from various sources including websites, many of which are unreliable. We aimed to create an informative, scientifically accurate and engaging website to educate patients about acupuncture for back pain and modify their beliefs in a way that might enhance its clinical effects.MethodsWe used psychological theory and techniques to design an evidence-based website, incorporating multimedia elements. We conducted qualitative “think aloud” audio-recorded interviews to elicit user views of the website. A convenience sample of ten participants (4 male; aged 21–64 years from the local community) looked at the website in the presence of a researcher and spoke their thoughts out loud. Comments were categorised by topic.ResultsThe website comprises 11 main pages and addresses key topics of interest to potential acupuncture patients, including beneficial and adverse effects, mechanisms of action, safety, practicalities, and patients’ experiences of acupuncture. It provides information through text, evidence summaries and audio-clips of four patients’ stories and two acupuncturists’ descriptions of their practice, and three short films. Evidence from the think aloud study was used to identify opportunities to make the website more informative, engaging, and user-friendly.ConclusionsUsing a combination of psychological theory and qualitative interviews enabled us to produce a user-friendly, evidence-based website that is likely to change patients’ beliefs about acupuncture for back pain. Before using the website in clinical settings it is necessary to test its effects on key outcomes including patients’ beliefs and capacity for making informed choices about acupuncture.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.