The design of test Q matrix can directly influence the classification accuracy of a cognitive diagnostic assessment. In this paper, we focus on Q matrix design when attribute hierarchies are known prior to test development. A complete Q matrix design is proposed and theorems are presented to demonstrate that it is a necessary and sufficient condition to guarantee the identifiability of ideal response patterns. A simulation study is also conducted to detect the effects of the proposed design on a family of conjunctive diagnostic models. The results revealed that the proposed Q matrix design is the key condition for guaranteeing classification accuracy. When only one type of item pattern in R matrix is missing from the associated test Q matrix, the related attribute-wise agreement rate will decrease dramatically. When the entire R matrix is missing, both the pattern-wise and attribute-wise agreement rates will decrease sharply. This indicates that the proposed procedures for complete Q matrix design with attribute hierarchies can serve as guidelines for test blueprint development prior to item writing in a cognitive diagnostic assessment.
Three widely-used self-report anxiety scales, including the Self-Rating Anxiety Scale (SAS), the Beck Anxiety Inventory (BAI), and the State Anxiety Inventory (S-AI), were used to simultaneously compare the psychometric properties via an item response theory (IRT) model with Chinese university students as the sample. Although these scales were probably to measure the same underlying construct, namely, anxiety, their psychometric properties were different. Results showed that the BAI’s measurement error was fewer than that of the other scales, with their anxiety severity ranging approximately from the 0.8 standard deviations below the mean to 3 standard deviations above the mean, while the S-AI’s measurement error was fewer than that of the other degrees of anxiety. The S-AI provided more information than the other scales when the student’s scale was less than approximately 0.8 standard deviations below the mean of anxiety severity. In general, the BAI showed better, for it provided more information than the other scales at the broadest range of anxiety severity. The SAS provided less information than the other scales at all anxiety severity range. In conclusion, BAI shows good psychometric quality. Finally, the three instruments were combined on a common scale by using IRT model and a conversion table was provided so as to achieve the transformation of each scale score.
Attribute hierarchy is a common assumption in the educational context, where the mastery of one attribute is assumed to be a prerequisite to the mastery of another one. The attribute hierarchy can be incorporated through a restricted Q matrix that implies the specified structure. The latent class–based cognitive diagnostic models (CDMs) usually do not assume a hierarchical structure among attributes, which means all profiles of attributes are possible in a population of interest. This study investigates different estimation methods to the classification accuracy for a family of CDMs when they are combined with a restricted Q-matrix design. A simulation study is used to explain the misclassification caused by an unrestricted estimation procedure. The advantages of the restricted estimation procedure utilizing attribute hierarchies for increased classification accuracy are also further illustrated through a real data analysis on a syllogistic reasoning diagnostic assessment. This research can provide guidelines for educational and psychological researchers and practitioners when they use CDMs to analyze the data with a restricted Q-matrix design and make them be aware of the potentially contaminated classification results if ignoring attribute hierarchies.
With the increasing prevalence of depression, creating a simple and precise tool for measuring depression is becoming more important. This study developed a computer adaptive testing for depression (CAT-Depression) from a Chinese sample. The depression item bank was constructed from a sample of 1,135 participants with or without depression using the Graded Response Model (GRM; Samejima, 1969). The final depression item bank with strict unidimensionality comprised 68 items, which had local independence, good item-fit, high discrimination, no differential item functioning (DIF), and each item measured at least one symptom of diagnostic criteria for depression in ICD-10. In addition, the mean IRT discrimination of the item bank reached 1.784, which clearly showed that the item bank of CAT-Depression was high-quality. Moreover, a simulation CAT study with real response data was conducted to investigate the characteristics, marginal reliability, criterion-related validity, and predictive utility (sensitivity and specificity) of CAT-Depression. The results revealed that the proposed CAT-Depression had acceptable and reasonable marginal reliability, criterion-related validity, and sensitivity and specificity.
To obtain accurate, valid, and rich information from the questionnaires for internet addiction, a diagnostic classification test for internet addiction (the DCT-IA) was developed using diagnostic classification models (DCMs), a cutting-edge psychometric theory, based on DSM-5. A calibration sample and a validation sample were recruited in this study to calibrate the item parameters of the DCT-IA and to examine the sensitivity and specificity. The DCT-IA had high reliability and validity based on both CTT and DCMs, and had a sensitivity of 0.935 and a specificity of 0.817 with AUC = 0.919. More important, different from traditional questionnaires, the DCT-IA can simultaneously provide general-level diagnostic information and the detailed symptom criteria-level information about the posterior probability of satisfying each symptom criterion in DMS-5 for each patient, which gives insight into tailoring individual-specific treatments for internet addiction.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.