Please cite as the following: Walkington, C., Clinton, V., & Shivraj, P. (2018). How readability factors are differentially associated with performance for students of different backgrounds when solving math word problems.
AbstractThe link between reading and mathematics achievement is well known, and an important question is whether readability factors in mathematics problems are differentially impacting student groups. Using 20 years of data from the National Assessment of Educational Progress and the Trends in International Mathematics and Science Study, we examine how readability factors -such as length, word difficulty, and pronouns -interact with student background characteristics -such as race/ethnicity, mathematics achievement, and socioeconomic status.Textual features that make problems more difficult to process appear to differentially negatively impact struggling students, while features that make language easier to process appear to differentially positively impact struggling students. It is critical that readability along various dimensions be considered when designing instruction and assessment.
INTRODUCTION:
As part of continuing certification, the American Board of Obstetrics and Gynecology (ABOG) diplomates complete article-based assessments annually. Diplomates read articles, respond to article questions, and must score above a certain threshold to demonstrate competence in their understanding. There is a paucity of evidence that diplomates retain the information after reading articles. The study objective was to evaluate the effectiveness of the two question types (understanding and application) at measuring diplomates’ retention of knowledge over time.
METHODS:
A sample of 344 diplomates who chose selected articles between January-March 2019 volunteered for the study. Diplomates completed article questions in a business-as-usual setting (with article access) and were assessed again during summer 2019 (without article access). Each participant received 2 questions (one of each type) from 5 of the selected articles, for a total of 10 questions. The timed summer assessment was used as a proxy for retention. Diplomates’ scores, by item type, from each assessment setting were compared using a t-test to determine whether information was retained from the articles.
RESULTS:
The information assessed by application items was retained at a significantly higher rate (80.58%) than understanding items (60.75%), as demonstrated by mean scores on the assessment (P<.05).
CONCLUSION:
Application items yielded a higher knowledge retention rate. Understanding items ensure diplomates comprehend the article while application questions ensure diplomates can generalize and apply their understanding from the article. A combination of both item types for article-based assessments is necessary and this study demonstrates that diplomates retain the information they read from articles.
Financial Disclosure Susan M. Ramin reports that she is the Associate Executive Director and fulltime employee of the ABOG and a non-voting member of the ABOG Board of Directors. She is also a volunteer member and the Secretary-Treasurer of the American Board of Medical Specialties (ABMS) Board of Directors for which she receives a yearly honorarium. George A. Macones serves a volunteer member of the ABOG Board of Directors and is the President of the Board of Directors. The non-employee directors of ABOG receive a yearly honorarium for their time and effort to serve as directors. George D. Wendel disclosed that he is the Executive Director and full-time employee of the ABOG and Secretary and non-voting member of the ABOG Board of Directors. He is also a Member of the American Board of Medical Specialties (ABMS) Board of Directors representing the ABOG and an ex-officio member of the Accreditation Council for Graduate Medical Education (ACGME) Review Committee for Obstetrics and Gynecology. The other authors did not report any potential conflicts of interest.
Many teachers struggle with interpreting formative assessment data. As a result, they are often unable to modify instruction that is responsive to student performance. To support valid interpretations and uses of assessments, test developers need to better understand teachers' ability to read and interpret score reports, their perception of the information that is being conveyed, and how they use the information to change instruction. The purpose of this manuscript is to report on a series of focus groups examining (a) how teachers interact with various score report features and (b) how these features support or hinder their ability to interpret and analyze formative assessment data. Results provide insights into the design of score reports that may facilitate teachers' ability to interpret and analyze data. Solutions to the roadblocks teachers encountered are proposed and generalized examples of how score reports can be designed to facilitate teachers' interpretations are provided. Findings from this study may help test developers anticipate challenges the intended audience could face when interpreting assessment results on score reports, and subsequently improve the quality of score reports to enhance teachers' interpretations and decision-making at the classroom level.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.