Identifying Naturally Occurring Direct Assessments of Social-Emotional Competencies: The Promise and Limitations of Survey and Assessment Disengagement Metadata

Abstract

Social-emotional learning (SEL) is gaining increasing attention in education policy and practice due to growing evidence that related constructs are strongly predictive of long-term academic achievement and attainment. However, the work of educators to support SEL is hampered by a lack of available, unbiased measures of related competencies. In this study we conducted a literature review to investigate whether assessment metadata (typically data relevant to how students behave on a test or survey) can provide information on SEL constructs. Implications of this new source of SEL data for practice, policy, and research are discussed.

Bi‐directional relations between behavioral problems and executive function: Assessing the longitudinal development of self‐regulation

“…It should also be noted that measures of both EF and behavioral problems showed some degree of stability and change in development over the years considered here. We found that EF measures tended to have stronger latent factor loadings than auto‐regressive paths, and we found that measures of behavioral problems had lower latent factor loadings when compared with EF, especially in the single measure model (see similar findings for social‐emotional skills in Soland et al., 2019). Yet, for both domains, auto‐regressive paths were largely statistically significant and substantively important in magnitude, suggesting that time‐varying factors can influence development in each domain.…”

Assessing Survey Satisficing: The Impact of Unmotivated Questionnaire Responding on Data Quality

“…In particular, respondents qualified as nonresponders as long as they omitted one item. With a data set this large, it seems inevitable that we inadvertently miscategorized some motivated students (e.g., those who missed an item by accident), thereby potentially misrepresenting student metadata (Soland et al, 2019). On the other hand, it is also possible that our straight-lining threshold of 10 items in a row excluded some respondents (e.g., those who straight-lined for nine items) who were putting forth suboptimal effort.…”

Heterogenita ve způsobu odpovídání na dotazníkové položky u českých žáků v PISA 2012 : analýza dle kraje, typu a velikosti školy

“…V současné zahraniční literatuře je míra chybějících odpovědí v dotazníku zkoumána i jako ukazatel míry angažovanosti respondentů při vyplňování dotazníku a jako potenciální alternativní indikátor nekognitivních dovedností žáků jako houževnatost (grit; viz např. Hitt et al, 2016;Soland et al, 2019). V PISA Zamarro et al (2019) nalezli rozdíly v této míře mezi různými zeměmi (např.…”