Evaluating the Psychometric Quality of Social Skills Measures: A Systematic Review

Abstract

IntroductionImpairments in social functioning are associated with an array of adverse outcomes. Social skills measures are commonly used by health professionals to assess and plan the treatment of social skills difficulties. There is a need to comprehensively evaluate the quality of psychometric properties reported across these measures to guide assessment and treatment planning.ObjectiveTo conduct a systematic review of the literature on the psychometric properties of social skills and behaviours measures for both children and adults.MethodsA systematic search was performed using four electronic databases: CINAHL, PsycINFO, Embase and Pubmed; the Health and Psychosocial Instruments database; and grey literature using PsycExtra and Google Scholar. The psychometric properties of the social skills measures were evaluated against the COSMIN taxonomy of measurement properties using pre-set psychometric criteria.ResultsThirty-Six studies and nine manuals were included to assess the psychometric properties of thirteen social skills measures that met the inclusion criteria. Most measures obtained excellent overall methodological quality scores for internal consistency and reliability. However, eight measures did not report measurement error, nine measures did not report cross-cultural validity and eleven measures did not report criterion validity.ConclusionsThe overall quality of the psychometric properties of most measures was satisfactory. The SSBS-2, HCSBS and PKBS-2 were the three measures with the most robust evidence of sound psychometric quality in at least seven of the eight psychometric properties that were appraised. A universal working definition of social functioning as an overarching construct is recommended. There is a need for ongoing research in the area of the psychometric properties of social skills and behaviours instruments.

Assessing Children’s Social Functioning and Reading Proficiency: A Systematic Review of the Quality of Educational Assessment Instruments Used in Norwegian Elementary Schools

“…Similar to the findings of previous studies (e.g., Cordier et al, 2015;Merrell, 2001; Statens beredning för medicinsk utvärdering [Swedish Council on Health Technology Assessment], 2014), our findings demonstrate that the documented educational assessment instruments that were reported as being used in schools to assess children's social functioning and reading proficiency varied with respect to the informal or formal structure of their assessment methods (e.g., open notes of teacher ratings versus criterion-based tests), the level of interactivity (e.g., static versus dynamic), whether they were summative or formative (e.g., assessment of learning versus assessment for learning), the assessment structure (e.g., presentation of the items to the testtaker), the response formats (e.g., selected or constructed items), and the item scoring (e.g., hand scoring versus computer-based scoring). However, our view is that assessments and interventions are often intertwined and that schools use what is readily available.…”

“…A number of studies show that social skills are required for the development of good social relations, emotional and academic engagement, and school motivation (Beauchamp & Anderson, 2010;Cordier et al, 2015;Gresham, 2007). Reading proficiency refers to the process of learning to decode words accurately and fluently and to comprehend the meaning of text (Hoover & Gough, 1990).…”

“…Whereas measures of social functioning are commonly based on informal teachers' ratings and students' self-reports, reading proficiency is measured based on summative formal tests or formative informal assessments and teacher ratings. Findings from systematic reviews show large variations in the methods that schools use to assess children's development in social skills and reading (Cordier et al, 2015;Floyd et al, 2015;Gotch & French, 2014;OECD, 2015 , 2015). Altogether, these reviews show that the educational assessment instruments used in schools vary with respect to a number of dimensions: (1) the level of informal or formal structure (e.g., open notes of teacher ratings versus criterion-based tests); (2) the level of interactivity (e.g., static versus dynamic); (3) whether the assessment is summative or formative (e.g., assessment of learning versus assessment for learning); (4) the assessment structure (e.g., presentation of the items to the test-taker); (5) the response formats (e.g., selected or constructed items); and (6) the item scoring (e.g., hand scoring versus computer-based scoring).…”

“…Reliability is crucial if an assessment is to be useful, as a test that is not reliable can never be a valid instrument (Thorndike & Thorndike-Christ, 2014). Reliability is commonly assessed in terms of internal consistency reliability (the degree to which different test items that probe the same construct produce similar results), test-retest reliability (stability over time), and inter-rater reliability (the degree to which different observers or raters agree).Recent systematic reviews evaluating the quality of educational assessment instruments have revealed a lack of studies of psychometric properties for many of the instruments used in schools (see, for instance, Cordier et al, 2015;Floyd et al, 2015;Gotch & French, 2014;Siddiq, Hatlevik, Olsen, Throndsen, & Scherer, 2016; Statens beredning för medicinsk utvärdering [Swedish Council on Health Technology Assessment], 2014). Although several of the reviewed measures demonstrated good psychometric qualities, many others showed weak or lacking evidence.…”

“…Although several of the reviewed measures demonstrated good psychometric qualities, many others showed weak or lacking evidence. In a review of 13 measures of social skills, Cordier et al (2015) found excellent reliability scores overall, but none of the measures were found to exhibit validity. Additionally, Floyd et al (2015) demonstrated that most of the 14 behaviour scales they reviewed had adequate or inadequate norming data.…”