Linear and nonlinear growth models for value-added assessment: an application to Spanish primary and secondary schools’ progress in reading comprehension

López-Martín, Esther; Kuosmanen, Timo; Gaviria, José Luis

doi:10.1007/s11092-014-9194-1

Cited by 6 publications

(9 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Within this pool of literature, researchers varied in their decisions regarding the appropriateness of including school effects in a model when the focus is on teacher accountability (e.g., Castellano, Rabe-Hesketh, & Skrondal, 2014; Ishii & Rivkin, 2009; Papay, 2011; Rothstein, 2009), the necessity of modeling differential effects based on student traits (e.g., Condie, Lefgren, & Sims, 2014; Jakubowski, 2008; Lockwood & McCaffrey, 2009; Reardon & Raudenbush, 2009), and the use of fixed (e.g., Bacher-Hicks, Kane, & Staiger, 2014; Dieterle, Guarino, Reckase, & Wooldridge, 2015; Rothstein, 2007) or random effects (e.g., Briggs & Weeks, 2009; Lopez-Martin et al, 2014). Researchers disagreed regarding appropriate model complexity, with some disagreement as to whether preferring simpler, and therefore more transparent, models are preferable to models that are more complex (e.g., Gorard, 2008; Kelly & Downey, 2010).…”

Section: Resultsmentioning

confidence: 99%

“…Although these issues deserve further study and have relevance to VA modeling, this study’s purpose is to examine methodological issues that may have an impact on VA-based accountability in K-12 schools. VA models may, in theory, include nonlinear growth models, and one paper in the identified pool of literature used a nonlinear approach with promising results (Lopez-Martin, Kuosmanen, & Gaviria, 2014). However, because the remaining papers use a linear approach, this literature review primarily focuses on the issues addressing linear VA models.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Value-Added Modeling and Educational Accountability

Everson

2016

Review of Educational Research

View full text Add to dashboard Cite

Value-added estimates of teacher or school quality are increasingly used for both high-and low-stakes accountability purposes, making understanding of their limitations critical. A review of the recent value-added literature suggests three concerns with the state of the research. First, the issues receiving the most research attention have not always been the concerns of greatest importance to theorists or critics. Second, there has been insufficient research on the interactions among various issues or assumption violations. Third, some of the big issues in value-added modeling have been challenging to address and may require educators to step back and answer some underlying philosophical questions about the nature of teacher and school quality.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Value-Added Modeling and Educational Accountability

Everson

2016

Review of Educational Research

View full text Add to dashboard Cite

show abstract

“…However, despite their practical relevance, there is currently no consensus on how to best estimate VA scores. The two most commonly used statistical models are linear regression and multilevel models ( Kurtz, 2018 ; Levy et al, 2019 ) and some researchers have applied non-linear models for the estimation of school VA scores ( Fitz-Gibbon, 1997 ; Lopez-Martin et al, 2014 ). An alternative approach to these classical models involves machine learning methods, which social sciences are drawing on more as larger and more complex data sets become increasingly available, as new types of data require new data analytical approaches.…”

Section: Discussionmentioning

confidence: 99%

“…Intuitively, most people would agree that learning does not happen linearly (e.g., as illustrated in this blog entry, McCrann, 2015 ). This is underlined by findings that at least in some cases, non-linear models fit the data better than linear models, implying that the typical linearity assumption might not be warranted ( Lopez-Martin et al, 2014 ). However, this does not necessarily mean that non-linear models are also more appropriate for the estimation of VA scores.…”

Section: Introductionmentioning

confidence: 99%

Contrasting Classical and Machine Learning Approaches in the Estimation of Value-Added Scores in Large-Scale Educational Data

et al. 2020

View full text Add to dashboard Cite

There is no consensus on which statistical model estimates school value-added (VA) most accurately. To date, the two most common statistical models used for the calculation of VA scores are two classical methods: linear regression and multilevel models. These models have the advantage of being relatively transparent and thus understandable for most researchers and practitioners. However, these statistical models are bound to certain assumptions (e.g., linearity) that might limit their prediction accuracy. Machine learning methods, which have yielded spectacular results in numerous fields, may be a valuable alternative to these classical models. Although big data is not new in general, it is relatively new in the realm of social sciences and education. New types of data require new data analytical approaches. Such techniques have already evolved in fields with a long tradition in crunching big data (e.g., gene technology). The objective of the present paper is to competently apply these "imported" techniques to education data, more precisely VA scores, and assess when and how they can extend or replace the classical psychometrics toolbox. The different models include linear and non-linear methods and extend classical models with the most commonly used machine learning methods (i.e., random forest, neural networks, support vector machines, and boosting). We used representative data of 3,026 students in 153 schools who took part in the standardized achievement tests of the Luxembourg School Monitoring Program in grades 1 and 3. Multilevel models outperformed classical linear and polynomial regressions, as well as different machine learning models. However, it could be observed that across all schools, school VA scores from different model types correlated highly. Yet, the percentage of disagreements as compared to multilevel models was not trivial and real-life implications for individual schools may still be dramatic depending on the model type used. Implications of these results and possible ethical concerns regarding the use of machine learning methods for decision-making in education are discussed.

show abstract

“…In quantitative research, statistical methods have been applied to investigate the necessary conditions for measuring academic achievement objectively, with the correct design and use of measurement instruments -for example, value added models and multilevel models from a longitudinal perspective (Blanco et al, 2009;López-Martín et al, 2014).…”

Section: The University Entrance Examinations In Spainmentioning

confidence: 99%

University Entrance Examinations in Spain: Using the Construct Comparability Approach to Analyze Standards Quality

et al. 2020

View full text Add to dashboard Cite

In recent years, important methodological attempts have been made to explore the comparability of examination standards, especially in the context of certifications and university entrance. The present study aimed to explore the use of a construct comparability approach through a comparative analysis of the academic scores on 15 subjects from Spanish University Entrance Examinations in the Valencian Community, with a sample of 22,996 students in the call of June 2018. We employed the Rasch partial credit model as an estimation method, counting each subject as the item of an instrument related to academic achievement. The results confirmed the unidimensionality assumption and the goodness of fit of the model in relation to all subjects, although no discrimination between high and low ability students was detected because of the lack of monotonicity of the score categories. We observed that the level of difficulty of the subjects was appropriate to the students' ability levels. Important conclusions have been drawn for the improvement of the standard qualification process, and future research directions have been proposed.

show abstract

Linear and nonlinear growth models for value-added assessment: an application to Spanish primary and secondary schools’ progress in reading comprehension

Cited by 6 publications

References 32 publications

Value-Added Modeling and Educational Accountability

Value-Added Modeling and Educational Accountability

Contrasting Classical and Machine Learning Approaches in the Estimation of Value-Added Scores in Large-Scale Educational Data

University Entrance Examinations in Spain: Using the Construct Comparability Approach to Analyze Standards Quality

Contact Info

Product

Resources

About