A Systematic Review and Psychometric Evaluation of Adaptive Behavior Scales and Recommendations for Practice

Floyd, Randy G.; Shands, Elizabeth I.; Alfonso, Vincent C.; Phillips, Jessica F.; Autry, Beth K.; Mosteller, Jessica A.; Skinner, Mary; Irby, Sarah M.

doi:10.1080/15377903.2014.979384

Cited by 34 publications

(26 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Thus, our findings support the conclusions of other systematic reviews of assessment measures, namely, that the psychometric evidence is weak or lacking (e.g., Cordier et al, 2015;Floyd et al, 2015;Gotch & French, 2014; Statens beredning för medicinsk utvärdering [Swedish Council on Health Technology Assessment], 2014). That said, we did not analyse the measurements' original data but rather reviewed the information and the documented materials of the instruments provided to us by the authors and publishers and obtained through the literature search.…”

Section: Discussionsupporting

confidence: 78%

“…Reliability is crucial if an assessment is to be useful, as a test that is not reliable can never be a valid instrument (Thorndike & Thorndike-Christ, 2014). Reliability is commonly assessed in terms of internal consistency reliability (the degree to which different test items that probe the same construct produce similar results), test-retest reliability (stability over time), and inter-rater reliability (the degree to which different observers or raters agree).Recent systematic reviews evaluating the quality of educational assessment instruments have revealed a lack of studies of psychometric properties for many of the instruments used in schools (see, for instance, Cordier et al, 2015;Floyd et al, 2015;Gotch & French, 2014;Siddiq, Hatlevik, Olsen, Throndsen, & Scherer, 2016; Statens beredning för medicinsk utvärdering [Swedish Council on Health Technology Assessment], 2014). Although several of the reviewed measures demonstrated good psychometric qualities, many others showed weak or lacking evidence.…”

mentioning

confidence: 99%

“…Whereas measures of social functioning are commonly based on informal teachers' ratings and students' self-reports, reading proficiency is measured based on summative formal tests or formative informal assessments and teacher ratings. Findings from systematic reviews show large variations in the methods that schools use to assess children's development in social skills and reading (Cordier et al, 2015;Floyd et al, 2015;Gotch & French, 2014;OECD, 2015 , 2015). Altogether, these reviews show that the educational assessment instruments used in schools vary with respect to a number of dimensions: (1) the level of informal or formal structure (e.g., open notes of teacher ratings versus criterion-based tests); (2) the level of interactivity (e.g., static versus dynamic); (3) whether the assessment is summative or formative (e.g., assessment of learning versus assessment for learning); (4) the assessment structure (e.g., presentation of the items to the test-taker); (5) the response formats (e.g., selected or constructed items); and (6) the item scoring (e.g., hand scoring versus computer-based scoring).…”

mentioning

confidence: 99%

See 2 more Smart Citations

Assessing Children’s Social Functioning and Reading Proficiency: A Systematic Review of the Quality of Educational Assessment Instruments Used in Norwegian Elementary Schools

Arnesen

Braeken

Ogden

et al. 2018

Scandinavian Journal of Educational Research

View full text Add to dashboard Cite

Social functioning and reading proficiency are critical for success in school and society. Therefore, identifying children with such problems is important. This study had 2 parts: first, a random sample of 234 elementary schools was surveyed about which instruments they use to assess reading proficiency and social functioning. Second, a systematic review of the quality of these instruments was conducted using international standards for examining the quality of assessment instruments. The survey showed that schools more often assessed and had more instruments available for reading than for social functioning. The systematic review of the assessment instruments used revealed that the psychometric qualities of many was weak or undocumented, while the dimensions of test material quality were generally good. The findings demonstrate a need for a more thorough examination of the psychometric properties of assessment instruments to be used in school. ARTICLE HISTORYReceived 9 February 2017 Accepted 15 November 2017 KEYWORDS Social functioning; reading proficiency; systematic review; educational assessment; psychometric quality Everyday decisions that impact children's social and academic development are based on information derived from a variety of educational assessments conducted in schools. Such decisions can influence children's curricula and whether a child receives additional support to prevent and ameliorate difficulties or receives a referral to educational psychology services for further diagnostics and special education. Thus, assessments are important for instructional decisions that may have a great impact on children's learning and wellbeing.It is reported that 15-20% of Norwegian children in Grades 1 to 10 are facing social emotional (i.e., anxiety, conduct disorders, depression) and/or academic (i.e., reading, math) difficulties that impact their academic success (Kunnskapsdepartementet [The Norwegian Ministry of Education], 2009, 2017). Moreover, 20% of children have a special requirement for more intensive support than their peers to succeed socially and/or academically (Kunnskapsdepartementet [The Norwegian Ministry of Education], 2017). Because social functioning and reading proficiency are strongly related to future life outcomes for students at risk, promoting such skills is crucial (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011;Gustafsson et al., 2010;OECD, 2015).Compared with the USA and the UK, Norway began using systematic assessments in schools relatively recently. Additionally, Norway, like many other European countries, faces a disadvantage in regard to the development of educational assessment instruments because it has a small population © 2018 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http:// creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduc...

show abstract

Section: Discussionsupporting

confidence: 78%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Assessing Children’s Social Functioning and Reading Proficiency: A Systematic Review of the Quality of Educational Assessment Instruments Used in Norwegian Elementary Schools

Arnesen

Braeken

Ogden

et al. 2018

Scandinavian Journal of Educational Research

View full text Add to dashboard Cite

show abstract

“…Notably, most of the instruments that have been validated focus more on problem behaviour than prosocial skills or broad social skills that are not as relevant in a school context (e.g. Crowe, Beauchamp, Catroppa and Anderson, 2011;Floyd et al, 2015).…”

Section: Social Skills Related To School Successmentioning

confidence: 99%

Validation of the elementary social behaviour assessment: teacher ratings of students’ social skills adapted to Norwegian, grades 1–6

Arnesen

Smolkowski

Ogden

et al. 2017

Emotional and Behavioural Difficulties

View full text Add to dashboard Cite

This study investigated the psychometric properties of the Elementary Social Behavior Assessment (ESBA), a teacher-report measure of students' social skills and academic engagement adapted for use in Norwegian schools. At two times, 8 weeks apart, 151 teachers rated 793 students in Grades 1-6 on the ESBA and the Social Skills Rating System Teacher's form (SSRS-T). Exploratory factor analysis with the first assessment suggested one or possibly two factors. Confirmatory factor analysis with the second assessment confirmed the one-or twofactor models but did not offer strong evidence for one over the other. The ESBA demonstrated good to excellent reliability. Strong concurrent and predictive correlations with the SSRS-T established criterion validity. The school and teacher levels explained only a limited amount of the total variance in ESBA scores, suggesting that teachers rate students quite similarly. The Norwegian-adapted ESBA paralleled the reliability and validity evidence from US samples.

show abstract

“…Reliability findings will also be reported for scores of each measure according to the different categories (i.e., standard error measurement, reliability/generalizability coefficients, test information functions) outlined in the Standards for Psychological and Educational Testing [ 46 ]. In synthesizing validity evidence data, other reviews have developed their own system of classifying measures according to various levels [ 35 , 50 ] or by assigning scores [ 51 , 52 ] based on the number of validity evidence sources for scores of a particular measure. To follow suit, measures will be categorized into four groups, based on the number of validity evidence sources established across studies (e.g., group 1 = 4 sources of validity evidence established, group 2 = 3 sources of validity evidence).…”

Section: Methodsmentioning

confidence: 99%

Measures assessing attributes of evidence-informed decision-making (EIDM) competence among nurses: a systematic review protocol

et al. 2018

View full text Add to dashboard Cite

BackgroundThere are growing professional expectations for nurses to engage in and develop competence in evidence-informed decision-making (EIDM) due to opportunities for improved client and community outcomes and provision of the highest quality of care. However, EIDM is underdeveloped, with low implementation rates among nurses. The use of indicators to assess EIDM performance has potential to encourage nurses’ engagement in EIDM through competence recognition and support assessment of strengths and competency gaps for individual nurses and organizations. Currently, the state of evidence regarding measures that assess EIDM competence attributes (i.e., knowledge, skills, beliefs/values, behaviors) among nurses is unknown. This systematic review aims to address this gap through a narrative synthesis of the characteristics and psychometric properties of EIDM competence measures.MethodsThe search strategy, developed in consultation with a Health Sciences Librarian, consists of online databases, contacting experts, hand searching reference lists, key journals, websites, conference proceedings, and grey literature. Studies will be included if the following criteria are met: (1) sample includes practicing nurses and data for nurses are reported separately; (2) conducted in any healthcare setting; (3) quantitative or mixed-methods design; (4) reports use or testing of a measure assessing EIDM competence attributes (i.e., knowledge, skills, attitudes/values, and/or behaviors); and (5) published in English. Screening will be conducted independently by two reviewers using a two-stage process: (1) title and abstract level; and (2) full-text level. Data extraction of study characteristics (e.g., sample, setting) will be conducted by a single reviewer and checked for accuracy by a second reviewer. Psychometric properties of acceptability, reliability, and validity evidence for each measure will be independently extracted by two reviewers. Data on measures will be synthesized narratively according to acceptability, number of validity evidence sources established, and reliability of scores. Data pertaining to population and healthcare setting will also be reported for each measure.DiscussionThis systematic review will provide a current understanding about the state of evidence with respect to EIDM competence measures in nursing to assist in determining potentially relevant and robust measures for use in different nursing practice settings.Systematic review registrationProtocol registered in PROSPERO Registration #: CRD42018088754Electronic supplementary materialThe online version of this article (10.1186/s13643-018-0849-8) contains supplementary material, which is available to authorized users.

show abstract

A Systematic Review and Psychometric Evaluation of Adaptive Behavior Scales and Recommendations for Practice

Cited by 34 publications

References 20 publications

Assessing Children’s Social Functioning and Reading Proficiency: A Systematic Review of the Quality of Educational Assessment Instruments Used in Norwegian Elementary Schools

Assessing Children’s Social Functioning and Reading Proficiency: A Systematic Review of the Quality of Educational Assessment Instruments Used in Norwegian Elementary Schools

Validation of the elementary social behaviour assessment: teacher ratings of students’ social skills adapted to Norwegian, grades 1–6

Measures assessing attributes of evidence-informed decision-making (EIDM) competence among nurses: a systematic review protocol

Contact Info

Product

Resources

About