The Social Media Index as an Indicator of Quality for Emergency Medicine Blogs: A METRIQ Study

Self Cite

Background: Online resources for emergency medicine (EM) trainees and physicians have variable quality and inconsistent coverage of core topics. In this first entry of the Society for Academic Emergency Medicine Systematic Online Academic Resource (SOAR) series, we describe the application of a systematic methodology to comprehensively identify, collate, and curate online content for topic-specific modules. Methods: A list of module topics and related terms was generated from the American Board of EmergencyMedicine's Model of the Clinical Practice of Emergency Medicine. The authors selected "renal and genitourinary" for the first module, which contained 35 terms; all MeSH headers and colloquial synonyms related to the topic and related terms were searched both within the 100 most impactful online educational websites per the Social Media Index and the FOAMsearch.net search engine. Duplicate entries, journal articles, images, and archives were excluded. The quality of each article was rated using the revised METRIQ (rMETRIQ) score.Results: The search yielded 13,058 online resources. After 12,717 items were excluded, 341 underwent quality assessment. All renal/genitourinary topics were covered by at least one resource. The median rMETRIQ score was 11 of 21 (interquartile range = 8-14). Calculus of urinary tract was most prominently featured with 60 posts. Thirty-four posts (10% of full-text screened FOAM articles) covering 12 core topics were identified as high quality (rMETRIQ ≥ 16). Conclusions:We demonstrated the feasibility of systematically identifying and curating FOAM resources for a specific EM topic and identified an overrepresentation of some subtopics. This curated list of resources may guide trainees, teacher recommendations, and resource producers. Further entries in the series will address other topics relevant to EM.T he well-documented growth and use, both formally (in residency curricula 1,2 ) and informally, 3-5 of free online medical education resources (FOAM) in emergency medicine (EM) have been remarkable. 3,4,6,7 However, resources are scattered across an enormous number of sites, of variable and difficult to evaluate quality, 8-10 and may not be at the appropriate level for all learners. 11 Additionally, various topics are covered extensively while others receive no or scant coverage 12 with no synthesis describing specific topics coverage, extent, level of intended learner, and quality.

Section: Methodsmentioning

confidence: 99%

Systematic Online Academic Resource (SOAR) Review: Renal and Genitourinary

Grock

Bhalerao

Chan

et al. 2019

Self Cite

“…Website viewers hailed from 217 countries, with approximately 35% from Canada, 30% from the United States, and 35% from the rest of the world. CanadiEM currently ranks ninth out of 148 in the June 2018 Social Media Index, which measures the impact and quality of emergency medicine and critical care websites.…”

Section: Discussionmentioning

confidence: 99%

CanadiEM: Accessing a Virtual Community of Practice to Create a Canadian National Medical Education Institution

Ting

Thoma

Luckett-Gatopoulos

et al. 2018

Self Cite

Background: The rise of free open-access medical education (FOAM) has led to a wide range of online resources in emergency medicine. Canadian physicians have been active contributors to FOAM.Objectives: We aimed to create a virtual community of practice that would serve as a national platform for collaboration, learning, and knowledge dissemination.Methods: CanadiEM was formed in 2016 from the merger of two Canadian websites and a podcast. Using a community-of-practice model, we introduced two training programs to support junior community members in becoming core editorial team members and employed asynchronous Web technologies to facilitate collaboration. We also introduced a coached peer review process and formed strategic alliances that aim to ensure a high quality of publication.Results: CanadiEM has become a portal for readers to access a broad range of FOAM content. The website has published 782 articles. Of these, 71 have undergone a coached peer review process. The website has received over 2.5 million page views from 217 countries, and the associated CRACKCast podcast has been downloaded over 750,000 times.Conclusions: CanadiEM has succeeded in building a national multi-interface dissemination network that fosters collaboration and knowledge sharing in emergency medicine while fostering junior digital scholars. The construction of a community of practice has been facilitated by quality assurance, training programs, and the use of asynchronous Web technologies. Ongoing challenges in sustainability include a volunteer workforce with high turnover.

“…This was a planned secondary analysis of data from the METRIQ study (http://metriqstudy.org), which recruited students, EM trainees, and EM attendings to rate the quality of 20 clinically oriented EM blog posts via an online survey between March 1, 2016, and June 1, 2016 . After rating five blog posts with the METRIQ‐8 score (outlined in Data Supplement S1, Table S1, available as supporting information in the online version of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1002/aet2.10376/full), participants rated the METRIQ‐8 score on usability and whether they would recommend it using 7‐point Likert scales (1 = strongly agree).…”

Section: Methodsmentioning

confidence: 99%

“…Unlike textbooks and journals, these online resources are rarely peer‐reviewed and critics raise concerns that learners are being misled . Supporting these concerns, the Medical Education Translational Resources: Impact and Quality (METRIQ) study found that gestalt evaluations of these resources were unreliable . This suggests that a systematized appraisal of these resources may be more appropriate …”

mentioning

confidence: 99%

The Revised METRIQ Score: A Quality Evaluation Tool for Online Educational Resources

Colmers

Krishnan

Chan

et al. 2019

Self Cite

Background: With the rapid proliferation of online medical education resources, quality evaluation is increasingly critical. The Medical Education Translational Resources: Impact and Quality (METRIQ) study evaluated the METRIQ-8 quality assessment instrument for blogs and collected feedback to improve it.Methods: As part of the larger METRIQ study, participants rated the quality of five blog posts on clinical emergency medicine topics using the eight-item METRIQ-8 score. Next, participants used a 7-point Likert scale and free-text comments to evaluate the METRIQ-8 score on ease of use, clarity of items, and likelihood of recommending it to others. Descriptive statistics were calculated and comments were thematically analyzed to guide the development of a revised METRIQ (rMETRIQ) score.Results: A total of 309 emergency medicine attendings, residents, and medical students completed the survey.The majority of participants felt the METRIQ-8 score was easy to use (mean AE SD = 2.7 AE 1.1 out of 7, with 1 indicating strong agreement) and would recommend it to others (2.7 AE 1.3 out of 7, with 1 indicating strong agreement). The thematic analysis suggested clarifying ambiguous questions, shortening the 7-point scale, specifying scoring anchors for the questions, eliminating the "unsure" option, and grouping-related questions. This analysis guided changes that resulted in the rMETRIQ score.Conclusion: Feedback on the METRIQ-8 score contributed to the development of the rMETRIQ score, which has improved clarity and usability. Further validity evidence on the rMETRIQ score is required.W ith increasing expansion of emergency medicine (EM) blogs and podcasts, residents frequently use these open educational resources to supplement and potentially replace traditional tools. 1-4 Unlike textbooks and journals, these online resources are rarely peer-reviewed 5-7 and critics raise concerns that learners From the