Opportunity-to-Learn Effects on Achievement: Analytical Aspects

Muthén, Bengt; Huang, Li-Chiao; Jo, Booil; Khoo, Siek-Toon; Goff, Ginger Nelson; Novak, John R.; Shih, Jeffrey C.

doi:10.3102/01623737017003371

Cited by 26 publications

(19 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Students have different opportunities to learn depending on the country in which they live, and it is reasonable to expect these different opportunities to learn lead to country-specific strengths and weaknesses (Klieme & Baumert, 2001;Muthén et al, 1995). In other words, students' performance in the skills or content represented in a test is related to their familiarity with or "exposure" to those same skills or content (Wolf, 1998).…”

Section: Types Of Reading Materialsmentioning

confidence: 99%

Reading subskill differences between students in Shanghai-China and the US: evidence from PISA 2009

Lei

Pace

2013

Educational Research and Evaluation

View full text Add to dashboard Cite

Based on different language systems and educational practices of their respective countries, hypotheses were made regarding how 15-year-old students from ShanghaiChina and the US might differ in the 5 reading subskills designated in the Programme for International Student Assessment (PISA) when they have the same overall reading ability (i.e., when their overall reading ability is controlled for). A multilevel analysis was conducted to test the hypotheses using the PISA 2009 reading dataset. When we controlled for students' overall reading ability, individual socioeconomic status (SES), and school mean SES, Shanghai-Chinese students performed significantly better in integrating and interpreting than US students. Further, when we controlled for students' overall reading ability and school mean SES, US students showed significantly higher performance in reading non-continuous texts than Shanghai-Chinese students, whereas US students showed significantly lower performance in reading continuous texts. The results of this study can inform reading instruction and learning in the 2 countries.

show abstract

Section: Types Of Reading Materialsmentioning

confidence: 99%

Reading subskill differences between students in Shanghai-China and the US: evidence from PISA 2009

Lei

Pace

2013

Educational Research and Evaluation

View full text Add to dashboard Cite

show abstract

“…The value of student reports of classroom processes is increasingly becoming a point of interest for researchers and policymakers. Research suggests that student reports can be reliable reflections of their experiences in the classroom and predict variation in achievement (Martinez, 2012;Met Project, 2012;Muthen et al, 1995) Student background and achievement data. For each student in the sample we collected their eighth-grade California Standards Science Test (CST-Science) scores from the district at the end of the school year.…”

Section: Instruments and Measuresmentioning

confidence: 99%

Measuring Classroom Assessment Practice Using Instructional Artifacts: A Validation Study of the QAS Notebook

Martinez

Borko

Stecher

et al. 2012

Educational Assessment

View full text Add to dashboard Cite

We report the results of a pilot validation study of the Quality Assessment in Science Notebook, a portfolio-like instrument for measuring teacher assessment practices in middle school science classrooms. A statewide sample of 42 teachers collected 2 notebooks during the school year, corresponding to science topics taught in the fall and spring. Each notebook was scored on 9 dimensions of assessment practice by 3 trained raters. Our analysis investigated the reliability and validity of notebook ratings, with particular emphasis on identifying key sources of error in the ratings. The results suggest that variation in teacher practice across notebooks (i.e., over time) was more important than idiosyncratic rater inconsistencies as a source of error in the scores. The validity results point to a dominant factor underlying the ratings and some predictive power of notebook ratings on student achievement. We discuss implications of the results for measuring assessment practice through artifacts, drawing conceptual and methodological lessons about our model of assessment practice, the consistency of raters, and the estimation of variance over time with classroom-based measures of instruction.Correspondence should be sent to José Felipe Martínez,

show abstract

“…To address this shortcoming of measuring coverage alone, OTL researchers usually obtain an indicator of topic centrality or emphasis. For example, importance can be measured directly by asking teachers to report the degree of emphasis placed on various topics, or by asking them to report how much instructional time (minutes, days, or class periods) was devoted to the topics (Muthén et al, 1995).…”

Section: Opportunity To Learn (Otl) Instructional Measuresmentioning

confidence: 99%

Instructional Sensitivity of a State's Standards-Based Assessment

D’Agostino

Welsh

Corson

2007

Educational Assessment

View full text Add to dashboard Cite

The accuracy of achievement test score inferences largely depends on the sensitivity of scores to instruction focused on tested objectives. Sensitivity requirements are particularly challenging for standards-based assessments because a variety of plausible instructional differences across classrooms must be detected. For this study, we developed a new method for capturing the alignment between how teachers bring standards to life in their classrooms and how the standards are defined on a test. Teachers were asked to report the degree to which they emphasized the state's academic standards, and to describe how they taught certain objectives from the standards. Two curriculum experts judged the alignment between how teachers brought the objectives to life in their classrooms and how the objectives were operationalized on the state test. Emphasis alone did not account for achievement differences among classrooms. The best predictors of classroom achievement were the match between how the standards were taught and tested, and the interaction between emphasis and match, indicating that test scores were sensitive to instruction of the standards, but in a narrow sense.

show abstract

Opportunity-to-Learn Effects on Achievement: Analytical Aspects

Cited by 26 publications

References 7 publications

Reading subskill differences between students in Shanghai-China and the US: evidence from PISA 2009

Reading subskill differences between students in Shanghai-China and the US: evidence from PISA 2009

Measuring Classroom Assessment Practice Using Instructional Artifacts: A Validation Study of the QAS Notebook

Instructional Sensitivity of a State's Standards-Based Assessment

Contact Info

Product

Resources

About