Web-Based Patient Educational Material on Osteosarcoma: Quantitative Assessment of Readability and Understandability

Abstract

Background Patients often turn to web-based resources following the diagnosis of osteosarcoma. To be fully understood by average American adults, the American Medical Association (AMA) and National Institutes of Health (NIH) recommend web-based health information to be written at a 6th grade level or lower. Previous analyses of osteosarcoma resources have not measured whether text is written such that readers can process key information (understandability) or identify available actions to take (actionability). The Patient Education Materials Assessment Tool (PEMAT) is a validated measurement of understandability and actionability. Objective The purpose of this study was to evaluate web-based osteosarcoma resources using measures of readability, understandability, and actionability. Methods Using the search term “osteosarcoma,” two independent Google searches were performed on March 7, 2020 (by AGS), and March 11, 2020 (by TRG). The top 50 results were collected. Websites were included if they were directed at providing patient education on osteosarcoma. Readability was quantified using validated algorithms: Flesh-Kincaid Grade Ease (FKGE), Flesch-Kincaid Grade-Level (FKGL). A higher FKGE score indicates that the material is easier to read. All other readability scores represent the US school grade level. Two independent PEMAT assessments were performed with independent scores assigned for both understandability and actionability. A PEMAT score of 70% or below is considered poorly understandable or poorly actionable. Statistical significance was defined as P≤.05. Results Two searches yielded 53 unique websites, of which 37 (70%) met the inclusion criteria. The mean FKGE and FKGL scores were 40.8 (SD 13.6) and 12.0 (SD 2.4), respectively. No website scored within the acceptable NIH or AHA recommended reading level. Only 4 (11%) and 1 (3%) website met the acceptable understandability and actionability threshold. Both understandability and actionability were positively correlated with FKGE (ρ=0.55, P<.001; ρ=0.60, P<.001), but were otherwise not significantly associated with other readability scores. There were no associations between readability (P=.15), understandability (P=.20), or actionability (P=.31) scores and Google rank. Conclusions Overall, web-based osteosarcoma patient educational materials scored poorly with respect to readability, understandability, and actionability. None of the web-based resources scored at the recommended reading level. Only 4 achieved the appropriate score to be considered understandable by the general public. Authors of patient resources should incorporate PEMAT and readability criteria to improve web-based resources to support patient understanding.

Patient education information material assessment criteria: A scoping review

“…Actionability (the degree to which readers will know what to do after reading the material) refers to how people can determine the appropriate next steps (e.g., when and where to seek care, which healthy behaviours to adopt) based on the information contained in a PEIM (Bui et al, 2018; Harris et al, 2018; Lange et al, 2015; Ottenhoff et al, 2018; S. K. Patel et al, 2015; Vishnevetsky et al, 2018) Actionability had been evaluated in conjunction with two other criteria: readability and understandability. Due to the low readability of patient education resources, the necessary education level for understanding them was higher than the fifth grade, resulting in low understandability; and as a result, actionability was also low (Abdul Rahman et al, 2020; Bui et al, 2018; Gillum, 2017; Guitton, 2015; Gulbrandsen et al, 2022; Harris et al, 2018; Lamb et al, 2022; Lipari et al, 2019; Magrath et al, 2022; McClure et al, 2016; Miles et al, 2021; Sideris et al, 2021; Vishnevetsky et al, 2018; Wang et al, 2022).…”

“…This criterion is evaluated using the Patient Education Materials Assessment Tool (PEMAT) (Lipari et al, 2019). This criterion was examined in fourteen studies (Abdul Rahman et al, 2020;Alwani et al, 2021;Bui et al, 2018;Gillum, 2017;Guitton, 2015;Gulbrandsen et al, 2022;Harris et al, 2018;Lamb et al, 2022;Lipari et al, 2019;Magrath et al, 2022;McClure et al, 2016;Miles et al, 2021;Sideris et al, 2021;Vishnevetsky et al, 2018;Wang et al, 2022). 9 Complexity Three criteria are used to determine the complexity of materials: structure, density, and dependency.…”

“…K. Patel et al, 2015;Vishnevetsky et al, 2018). This criterion was evaluated in 23 studies (Abdul Rahman et al, 2020;Alshehri & Joury, 2019;Alwani et al, 2021;Arslan et al, 2020;Bacher et al, 2020;Balakrishnan, Chandy, Hseih, et al, 2016;Bui et al, 2018;Edmunds et al, 2014;Georgsson & Carlsson, 2022;Gillum, 2017;Guitton, 2015;Gulbrandsen et al, 2022;Harris et al, 2018;Hughes et al, 2019;Lake et al, 2007;Lamb et al, 2022;Lipari et al, 2019;Mac et al, 2020;Magrath et al, 2022;Mani et al, 2021;McClure et al, 2016;Miles et al, 2021;O'Sullivan et al, 2020;Sideris et al, 2021;Townsend, 2011;Unaka et al, 2017;van Ballegooie & Hoang, 2021;Vishnevetsky et al, 2018;Wang et al, 2022).…”

“…(Continues) Bui et al, 2018;Gillum, 2017;Guitton, 2015;Gulbrandsen et al, 2022;Harris et al, 2018;Lamb et al, 2022;Lipari et al, 2019;Magrath et al, 2022;McClure et al, 2016;Miles et al, 2021;Sideris et al, 2021;Vishnevetsky et al, 2018;Wang et al, 2022).…”

“…H. Garcia et al, 2014;S. F. Garcia et al, 2010;Georgsson & Carlsson, 2022;Ghodasra et al, 2018;Gollapalli et al, 2022;Griffin et al, 2003;Griffin et al, 2006;Grose et al, 2022;Guan et al, 2018;Guitton, 2015;Gulbrandsen et al, 2022;Habeeb, 2021;Handler et al, 2021;Hansberry et al, 2015;Hansberry et al, 2018;Hansberry, John, et al, 2014;Harris et al, 2018;Hill-Briggs et al, 2012;Hoang & van Courtney, 2022;Hosseinzadeh et al, 2021;Hunter et al, 2012;Iacovetto et al, 2014;Irwin et al, 2021;Johnson et al, 2016;Joseph et al, 2020;Kapoor et al, 2017;Kasabwala et al, 2012;Kasabwala et al, 2013;Kecojevic et al, 2020;Kelly et al, 2016;Khazaal et al, 2008;Khurana et al, 2003;Kim et al, 2022;Kirkpatrick & Mohler, 1999;Koo & Yap, 2017;Krass et al, 2002;Kruse et al, 2021;Lake et al, 2007;Lamb et al, 2022;Lavin et al, 2022;K. C. Lee et al, 2019;…”