Multiple measures, such as multiple content domains or multiple types of performance, are used in various testing programs to classify examinees for screening or selection. Despite the popular usages of multiple measures, there is little research on classification consistency and accuracy of multiple measures. Accordingly, this study introduces an approach to estimate classification consistency and accuracy indices for multiple measures under four possible decision rules: (1) complementary, (2) conjunctive, (3) compensatory, and (4) pairwise combinations of the three. The current study uses the IRT-recursive-based approach with the simple-structure multidimensional IRT model (SS-MIRT) to estimate the classification consistency and accuracy for multiple measures. Theoretical formulations of the four decision rules with a binary decision (Pass/Fail) are presented. The estimation procedures are illustrated using an empirical data example based on SS-MIRT. In addition, this study applies the estimation procedures to the unidimensional IRT (UIRT) context, considering that UIRT is practically used more. This application shows that the proposed procedure of classification consistency and accuracy could be used with a UIRT model for individual measures as an alternative method of SS-MIRT.Classification based on educational and psychosocial assessments directly impacts examinees' future learning (e.g., high school equivalency credential) or career success (e.g., certificate or licensure examination). An inaccurate classification would cause severe negative consequences. For instance, students who are incorrectly classified as "Fail" might not be able to have access to higher education. In addition, if a medical certificate is issued to a candidate who is not qualified, that candidate is more likely to cause medical accidents. Thus, classifying examinees accurately is important.Many high-stakes assessments involve the use of multiple measures to classify examinees; that is, multiple content domains or multiple types of performances are used to measure required abilities and skills accurately (Baker, 2003;Brookhart, 2009;Douglas & Mislevy, 2010;Henderson-Montero et al., 2003). The HiSET® exam (Educational Testing Service, 2018), which is a high school equivalency credential exam, is an example that consists of multiple domains-language arts-reading, language art-writing, mathematics, science, and social studies. Multiple types of performances are also commonly used in classification contexts. The Uniform Bar Examination
Historically, there has been concern about students losing reading ability over extended breaks from school, commonly in the summer, but studies of this phenomenon have produced inconsistent results. We applied exploratory visual analysis of multiple datasets to examine whether students in Grades K‐5 appear to lose or improve in various reading abilities over the summer and across consecutive school years. Archival data were obtained on students of different U.S. school districts who did not participate in a formal summer reading programme. Data were disaggregated by groups considered most vulnerable to summer loss: those from economically disadvantaged backgrounds and those identified with disabilities. Given the variety of measure types and scores, we centred scores on each measure's cut point for proficiency in a particular grade level to depict how students' scores deviated from the proficiency classifications before and after a summer break. Overall, students' scores relative to the benchmark appeared on average to have maintained or improved, and there was no observed accumulated decrement in reading performance across years. For anomalous instances of summer loss, we offer possible alternative explanations such as measurement artefacts and unrehearsed learning. Visual analysis of the datasets suggested that summer breaks were not associated with systematic losses of students' reading ability, even among those considered most vulnerable to the phenomenon. However, available assessments and benchmarks are not designed to measure summer learning specifically, and little is known about the kinds of literacy experiences students not in formal programmes might be having. Thus, more research on summer maturation and degeneration is warranted.
Latent class analysis (LCA) has been applied in many research areas to disentangle the heterogeneity of a population. Despite its popularity, its estimation method is limited to maximum likelihood estimation (MLE), which requires large samples to satisfy both the multivariate normality assumption and local independence assumption. Although many suggestions regarding adequate sample sizes were proposed, researchers continue to apply LCA with relatively smaller samples. When covariates are involved, the estimation issue is encountered more. In this study, we suggest a different estimating approach for LCA with covariates, also known as latent class regression (LCR), using a fuzzy clustering method and generalized structured component analysis (GSCA). This new approach is free from the distributional assumption and stable in estimating parameters. Parallel to the three-step approach used in the MLE-based LCA, we extend an algorithm of fuzzy clusterwise GSCA into LCR. This proposed algorithm has been demonstrated with an empirical data with both categorical and continuous covariates. Because the proposed algorithm can be used for a relatively small sample in LCR without requiring a multivariate normality assumption, the new algorithm is more applicable to social, behavioral, and health sciences.
Fuzzy clustering has been broadly applied to classify data into K clusters by assigning membership probabilities of each data point close to K centroids. Such a function has been applied into characterizing the clusters associated with a statistical model such as structural equation modeling. The characteristics identified by the statistical model further define the clusters as heterogeneous groups selected from a population. Recently, such statistical model has been formulated as fuzzy clusterwise generalized structured component analysis (fuzzy clusterwise GSCA). The same as in fuzzy clustering, the clusters are enumerated to infer the population and its parameters within the fuzzy clusterwise GSCA. However, the identification of clusters in fuzzy clustering is a difficult task because of the data-dependence of classification indexes, which is known as a cluster validity problem. We examined the cluster validity problem within the fuzzy clusterwise GSCA framework and proposed a new criterion for selecting the most optimal number of clusters using both fit indexes of the GSCA and the fuzzy validity indexes in fuzzy clustering. The criterion, named the FIT-FHV method combining a fit index, FIT, from GSCA and a cluster validation measure, FHV, from fuzzy clustering, performed better than any other indices used in fuzzy clusterwise GSCA.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.