Getting it ‘better’: the importance of improving background questionnaires in international large‐scale assessment

Rutkowski, Leslie; Rutkowski, David

doi:10.1080/00220272.2010.487546

Cited by 61 publications

(48 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Within ILSA research each of these poses its own set of challenges. For example, missing rates and disagreement between students and parents on identical, policy relevant variables has been shown to be high (see Rutkowski and Rutkowski 2010). Similarly, scale reliabilities can vary widely between countries, from high to unacceptably low (see Rutkowski and Rutkowski 2013).…”

Section: Resultsmentioning

confidence: 99%

Causal inferences with large scale assessment data: using a validity framework

Rutkowski

Delandshere

2016

Large-scale Assess Educ

Self Cite

View full text Add to dashboard Cite

Section: Resultsmentioning

confidence: 99%

Causal inferences with large scale assessment data: using a validity framework

Rutkowski

Delandshere

2016

Large-scale Assess Educ

Self Cite

View full text Add to dashboard Cite

“…Otros ponen de manifiesto la opacidad que caracteriza a este tipo de evaluaciones, a pesar de los esfuerzos de los responsables de PISA por poner a disposición de la sociedad manuales, guías y bases de datos (Domínguez, Vieira & Vidal, 2012;Pérez y Soto, 2011;Rutkowski & Rutkowski, 2010).…”

Section: │2unclassified

El impacto del Informe PISA en la sociedad española: el caso de la prensa escrita

González-Mayorga

García

Aller

2017

RELIEVE

View full text Add to dashboard Cite

ResumenEn la última década los resultados de los diferentes Informes PISA han avivado el debate sobre la calidad de nuestro sistema educativo, repercutiendo en la opinión pública y en la toma de decisiones en el plano político. En el presente estudio se analiza el impacto que han tenido los resultados de PISA en la sociedad española a través de las noticias publicadas en el periódico «El País» desde el 2001 al 2014, atendiendo a las siguientes cuestiones: a) los subgéneros periodísticos utilizados para hablar de PISA y su autoría; b) la evolución de las noticias desde PISA 2001 hasta PISA 2012; y c) la frecuencia de uso de las competencias evaluadas por PISA como argumento informativo. Los resultados muestran la gran trascendencia de la evaluación PISA en prensa, sobre todo desde la edición del Informe correspondiente al año 2003 y reflejado en su mayoría en artículos de carácter informativodescriptivo. Por otro lado, se evidencia el tratamiento que hacen los medios de comunicación escrita a las competencias matemática, científica y lectora, dando una mayor relevancia a esta última en los últimos años. Por último, se concluye que la prensa escrita ha sido un gran contribuyente a la hora de trasladar los resultados obtenidos en PISA a la opinión pública, a pesar del uso inadecuado que se hace en ocasiones de los datos sin tener en cuenta las limitaciones del propio estudio.Palabras clave: Impacto, PISA, competencias, prensa, educación y medios, debate, opinión pública, uso de la información. Fecha de recepción 2016 October 10 Fecha de aprobación 2017 February 8 Fecha de publicación 2017 February 9 AbstractIn the last decade, the results of the Programme for International Student Assessment (PISA) have increased the debate about the quality of the Spanish educational system, reverberating in the public opinion and the policy decision making. This study aims to analyse the impact the results of PISA in the Spanish society through the articles which have appeared in «El País» (the non-sport newspaper with the biggest circulation in Spain) from 2001 to 2014, attending to the following issues: a) the press subgenres used to talk about PISA and its authorship; b) the evolution of news from PISA 2001 to PISA 2012; and c) the frequency of use of the skills assessed by PISA as an information argument. The results of our study show the great importance of PISA results in press, especially since the 2003 edition, which was mostly reflected in descriptive and informative texts. On the other hand, the use that media makes of the competences in mathematics, science and reading is evidenced, giving a greater relevance to this last one in recent years. Finally, we conclude that the press has been a great contributor when it comes to the transfer of PISA results to the public opinion, in spite the inappropriate use that is sometimes made of data without considering the limitations of the study.

show abstract

“…Because the constructs come largely from western research and empirical evidence based on previous TIMSS research (e.g. Rutkowski and Rutkowski 2010;Metsämuuronen 2012a, b), we hypothesized that the reliability estimates would be lower in our sub-sample. Moreover, we expected the reliability of the confidence and like mathematics constructs to be affected by the method effect associated with the negatively worded item (e.g., Roszkowski and Soven 2010;Woods 2006;Brown 2015).…”

Section: Reliabilitymentioning

confidence: 99%

“…Studies have shown that the scales (e.g., mathematics self-concept) that had been used in PISA and TIMSS studies were less reliable in East Asia, the Middle East, and some parts of Europe, when compared with North America, where the scales were originally constructed Metsämuuronen 2012a, b;Rutkowski and Rutkowski, 2010). Other studies (Bofah andHannula 2014, 2015;Tuohilampi et al 2013) have also reported similar findings with other imported instruments-the students' views of mathematics instruments (VOM).…”

Section: Motivational Constructs In Affect: Cross Cultural Perspectivementioning

confidence: 99%

TIMSS data in an African comparative perspective: Investigating the factors influencing achievement in mathematics and their psychometric properties

Bofah

Hannula

2015

Large-scale Assess Educ

View full text Add to dashboard Cite

Relationships among motivational constructs from the 2011 Trends in International Mathematics and Science Study (TIMSS 2011) were investigated for eight-graders in all the five participating African countries, representing 38,806 (49 % girls). First, we investigated the psychometric properties (factor structure, reliabilities, method effect, and measurement invariance-country and gender) of the mathematics motivational constructs across the five educational systems. There was empirical support for the multidimensionality of the construct and the TIMSS 2011 motivational construct was largely invariant across cultures. Furthermore, a series of confirmatory factor analyses revealed that there is a need to control method effects associated with negatively worded items in the measurement model. There was support suggesting that in many cultures responses to negatively worded items are systematically different. The factor structures and reliabilities (i.e., confidence and the like mathematics scales) were affected by negatively worded items. Second, the relationships between the constructs, achievements and background variables such as parental education, gender and students' educational aspirations were investigated. We identified several significant relationships between self-belief and mathematics achievement. Differences in the latent mean achievement and the motivational construct were similar to those that have been described in the literature as "paradoxical" and "perplexing". Nations with high mathematics achievement seem to have students with more negative mathematics self-belief. Some results extend, whereas others refute the findings of previous research. For instance, the relationship between students' mathematics confidence and mathematics achievement was lower than the relationship between the value of mathematics and achievement in some countries and it was the reverse in others. However, consistent with cultural stereotypes, boys rated their mathematics competence higher than girls. The findings are discussed with reference to implications for cross-cultural research and practice.

show abstract

Getting it ‘better’: the importance of improving background questionnaires in international large‐scale assessment

Cited by 61 publications

References 16 publications

Causal inferences with large scale assessment data: using a validity framework

Causal inferences with large scale assessment data: using a validity framework

El impacto del Informe PISA en la sociedad española: el caso de la prensa escrita

TIMSS data in an African comparative perspective: Investigating the factors influencing achievement in mathematics and their psychometric properties

Contact Info

Product

Resources

About