Value-Added Models (VAMs): Caveat Emptor

Pivovarova, Margarita; Amrein‐Beardsley, Audrey; Broatch, Jennifer

doi:10.1080/2330443x.2016.1164641

Cited by 10 publications

(8 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consistent unreliability in the measures used to assess teachers, strongly limits validity. Test-based accountability systems, especially VAM-based accountability, ought to be avoided (Pivovarova et al, 2016). Nevertheless, politicians and policy makers seem partial to test-based models of teacher evaluation, even when researchers point out that the amount of variance in student test scores that is attributable to their teachers is negligible (American Statistical Association, 2014).…”

Section: Reliability Of Quantitative and Observational Evaluationsmentioning

confidence: 99%

The value of student feedback in open forums: A natural analysis of descriptions of poorly rated teachers

Valcarcel

Holmes

Berliner

et al. 2021

EPAA

View full text Add to dashboard Cite

In this paper we used natural language processing to review hundreds of thousands of negative student reviews of their teachers submitted to the website RateMyTeacher.com. Our analysis identified several issues raised by students when rating teachers poorly, which adds to the literature that defines “bad teachers” from the student perspective. We also identify the language students used to describe these issues and notice a clear distinction between the language used to address teaching-related complaints and behavior perceived as unfit for teachers. We argue that digital forums can be valuable tools for schools and conclude with suggestions and examples of the type of policies that may be derived from this type of analysis of a digital forum for students.

show abstract

Section: Reliability Of Quantitative and Observational Evaluationsmentioning

confidence: 99%

The value of student feedback in open forums: A natural analysis of descriptions of poorly rated teachers

Valcarcel

Holmes

Berliner

et al. 2021

EPAA

View full text Add to dashboard Cite

show abstract

“…Again, value-added measures, for example, can be analysed and modelled in multiple ways, leading to varying outcomes; this can result in negative associations between achievement and achievement gains (Ready, 2013). There is a sense, including on the basis of the statistics that inform such measures, that such measures should be used cautiously, particularly for significant and consequential decision-making (Pivovarova et al, 2016).…”

Section: Teacher Learning and Engaging With Datamentioning

confidence: 99%

Data for learning? Confirming and contesting performative practices of data governance

Hardy

Hamid

Reyes

2018

Global Studies of Childhood

View full text Add to dashboard Cite

This article examines evidence of teachers' work and learning in one school setting in the northern regions of Queensland, Australia, revealing how globalized performative practices that circulate around the collection and use of data in schooling settings are both confirmed and contested. Drawing upon the literature on the nature of accountability, particularly in relation to educational governance, and alternative theorizing of educational practice as praxis, the research reveals how teachers' work and learning are heavily implicated in the development and perpetuation of reductive, performative conceptions of education, even as teachers seek to foster a more educative disposition. The research draws upon interviews and meeting transcripts of teachers seeking to enhance their own learning for student learning. The article reveals that even as teachers are actively involved in developing and analysing a plethora of data to foster their own learning for enhanced student learning, they also struggle to do so in a context that ascribes particular standardized forms of school, regional and national data as the data of most value. At the same time, the article describes how more sustainable practices do exist, and serve as examples of forms of praxis that challenge more reductive, performative conceptions of learning in schools more broadly. ArticleIn this article, I focus upon the nature of teacher learning under current policy conditions. As such, teacher learning is understood as all the practices and processes in which teachers engage as part of their work, including the formal and informal activities in which they participate at and beyond the school level, and that contribute to informing understandings of their work; this includes both 'formal' and 'informal' modes of professional development (sometimes described as 'PD'; Hardy, 2012). Increasingly, teachers' learning is made sense of through the lens of specific forms of data, and what Anagnostopoulos et al. (2013) refer to as the substantial 'infrastructure of accountability' that enables the collection, analysis and engagement with such data. Henig (2013) argues the motivations behind these initiatives are not new, and include a trust and belief in the ability to develop institutions that can operationalize seemingly productive organizational incentives rather than being beholden to individual whims and self-interest. Such motivations also include trust in technical expertise, and that such expertise can provide solutions to seemingly intractable problems. Referring to the US context, Henig (2013) argues a third motivation is the belief that politics can be corrupting, and that systems need to be put into place that are not beholden to partisan approaches or philosophies.However, as Henig (2013) also argues, while the motivations behind the development of new forms of managing and monitoring evidence of learning are not new, the specific technologies deployed to do so, are. Such an infrastructure refers to the myriad forms of technology and technical expert...

show abstract

“…Much scholarship exists exploring the technical dimensions of VAM (e.g., McCaffrey, Lockwood, Koretz, & Hamilton, 2003), as well as its strengths, limitations, misinterpretations, and potential misuses (Baker et al, 2010;Pivovarova, Broatch, & Amrein-Beardsley, 2014). Work has also been done to articulate and scrutinize the assumptions that are necessary in order to use the models in the measurement of teacher effectiveness (American Statistical Association [ASA], 2014; Braun, 2005;Haertel, 2013).…”

Section: Value-added Modeling and The Measurement Of Teacher Effectivenessmentioning

confidence: 99%

Expertise Gaps in Value-Added Modeling

Tobiason

2014

Bulletin of Science, Technology & Society

View full text Add to dashboard Cite

This study introduces the notion of an "expertise gap," a mismatch between researcher expertise and research content. I show that these gaps can exist and warrant our attention in research endeavors that involve multidisciplinarity embedded in complex research designs. Writing as an education researcher concerned with educator quality, I show that several expertise gaps are embedded in a particular statistical approach to the measurement of teacher effectiveness: the use of value-added models. My analysis lays out the basic argument structure behind these models, locates eight consequential assumptions that researchers are obligated to make, and considers the kind of disciplinary expertise that these assumptions call for. I argue for the importance of illuminating expertise gaps and studying how they shape public understanding of science and technology.

show abstract

Value-Added Models (VAMs): Caveat Emptor

Cited by 10 publications

References 65 publications

The value of student feedback in open forums: A natural analysis of descriptions of poorly rated teachers

The value of student feedback in open forums: A natural analysis of descriptions of poorly rated teachers

Data for learning? Confirming and contesting performative practices of data governance

Expertise Gaps in Value-Added Modeling

Contact Info

Product

Resources

About