Challenges to the Use of Artificial Neural Networks for Diagnostic Classifications with Student Test Data

Briggs, Derek C.; Circi, Ruhan

doi:10.1080/15305058.2017.1297816

Cited by 8 publications

(9 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because of the large number of parameters contained in the deep learning structure, the random initialization of parameters may affect the optimization when the training sample size is not large enough. Thus, one concern of using ANNs/deep learning techniques for psychometrics is that using a feature extracted by deep learning through a single training is risky as it is sensitive to the starting points of the parameters (Briggs & Circi, 2017). To solve this concern, we conducted 100 DFN trainings individually, produced ability estimates for each training, and then averaged the results as the final estimates of ability for anchor students.…”

Section: Analysis and Resultsmentioning

confidence: 99%

Semisupervised Learning Method to Adjust Biased Item Difficulty Estimates Caused by Nonignorable Missingness in a Virtual Learning Environment

Xue

Huggins-Manley

Leite

2021

Educational and Psychological Measurement

View full text Add to dashboard Cite

In data collected from virtual learning environments (VLEs), item response theory (IRT) models can be used to guide the ongoing measurement of student ability. However, such applications of IRT rely on unbiased item parameter estimates associated with test items in the VLE. Without formal piloting of the items, one can expect a large amount of nonignorable missing data in the VLE log file data, and this is expected to negatively affect IRT item parameter estimation accuracy, which then negatively affects any future ability estimates utilized in the VLE. In the psychometric literature, methods for handling missing data have been studied mostly around conditions in which the data and the amount of missing data are not as large as those that come from VLEs. In this article, we introduce a semisupervised learning method to deal with a large proportion of missingness contained in VLE data from which one needs to obtain unbiased item parameter estimates. First, we explored the factors relating to the missing data. Then we implemented a semisupervised learning method under the two-parameter logistic IRT model to estimate the latent abilities of students. Last, we applied two adjustment methods designed to reduce bias in item parameter estimates. The proposed framework showed its potential for obtaining unbiased item parameter estimates that can then be fixed in the VLE in order to obtain ongoing ability estimates for operational purposes.

show abstract

Section: Analysis and Resultsmentioning

confidence: 99%

Semisupervised Learning Method to Adjust Biased Item Difficulty Estimates Caused by Nonignorable Missingness in a Virtual Learning Environment

Xue

Huggins-Manley

Leite

2021

Educational and Psychological Measurement

View full text Add to dashboard Cite

show abstract

“…Because of the large number of parameters contained in the deep learning structure, the random initialization of parameters may impact the optimization when the training sample size is not large enough. Thus, one concern of using ANNs for CDM is that using the feature extracted by deep learning through a single training is risky or sensitive to the starting points of the parameters (Briggs and Circi, 2017 ). Cui et al ( 2016 ) only set a maximum number of iterations (e.g., 10,000) to stop training the supervised learning ANN in their research study.…”

Section: Methodsmentioning

confidence: 99%

“…However, in both research studies, the unsupervised learning ANNs cannot yield comparable classification results compared with the DCMs, especially when the diagnostic quality of the assessment was not high. In addition, the ANNs methods produced very unstable and unappreciated estimation unless a great deal of care was taken to conduct sensitivity analyses (Briggs and Circi, 2017 ).…”

Section: Introductionmentioning

confidence: 99%

A Semi-supervised Learning-Based Diagnostic Classification Method Using Artificial Neural Networks

Xue

Bradshaw

2021

Front. Psychol.

View full text Add to dashboard Cite

The purpose of cognitive diagnostic modeling (CDM) is to classify students' latent attribute profiles using their responses to the diagnostic assessment. In recent years, each diagnostic classification model (DCM) makes different assumptions about the relationship between a student's response pattern and attribute profile. The previous research studies showed that the inappropriate DCMs and inaccurate Q-matrix impact diagnostic classification accuracy. Artificial Neural Networks (ANNs) have been proposed as a promising approach to convert a pattern of item responses into a diagnostic classification in some research studies. However, the ANNs methods produced very unstable and unappreciated estimation unless a great deal of care was taken. In this research, we combined ANNs with two typical DCMs, the deterministic-input, noisy, “and” gate (DINA) model and the deterministic-inputs, noisy, “or” gate (DINO) model, within a semi-supervised learning framework to achieve a robust and accurate classification. In both simulated study and real data study, the experimental results showed that the proposed method could achieve appreciated performance across different test conditions, especially when the diagnostic quality of assessment was not high and the Q-matrix contained misspecified elements. This research study is the first time of applying the thinking of semi-supervised learning into CDM. Also, we used the validating test to choose the appropriate parameters for the ANNs instead of using typical statistical criteria.

show abstract

“…e traditional cognitive psychology model only obtains the evaluation value of the cognitive ability model obtained after learning a certain chapter through modelling [19]. To achieve the comprehensive evaluation value of the entire training activity, the traditional cognitive psychology model is used.…”

Section: Complexitymentioning

confidence: 99%

Relationship between Cognitive Learning Psychological Classification and Neural Network Design Elements

Yang

Yong

et al. 2021

Complexity

View full text Add to dashboard Cite

This article first analyzes the research background of the design elements of cognitive psychology and neural networks at home and abroad, roughly understands the research status and research background of these two courses at home and abroad, and discusses the application of cognitive psychology to neural networks. The design method has not yet formed a systematic theoretical system. Then, a systematic theoretical analysis of the research in this article is carried out to analyze the relationship between the various characteristics of cognitive psychology and the design elements of the neural network, and it uses these relationships to guide the design practice. Second, it analyzes the relationship between the influence and interaction of cognitive psychology on neural network design and connects cognitive psychology with neural network design. Finally, according to the theoretical analysis and research of the system, the application of cognitive psychology in neural network design, design practice, and the relationship between the two are systematically reviewed. Through the exploratory research on cognitive psychology in neural network design, we can see that the combination of neural network design and psychology, art aesthetics, and other cross-disciplinary and multidisciplinary research is necessary, which can promote the scientific and technological progress of neural network design in the context of the information age and the improvement of public mental health. Under the background of the era in which the neural network design becomes the link between people's emotions and culture, we must fully understand the essential role of each element in neural network design and build a design concept based on cognitive psychology and emotional experience. It is hoped that the content of this topic can provide a certain reference value for the future development of neural network design and cognitive psychology and clarify the new development direction.

show abstract

Challenges to the Use of Artificial Neural Networks for Diagnostic Classifications with Student Test Data

Cited by 8 publications

References 16 publications

Semisupervised Learning Method to Adjust Biased Item Difficulty Estimates Caused by Nonignorable Missingness in a Virtual Learning Environment

Semisupervised Learning Method to Adjust Biased Item Difficulty Estimates Caused by Nonignorable Missingness in a Virtual Learning Environment

A Semi-supervised Learning-Based Diagnostic Classification Method Using Artificial Neural Networks

Relationship between Cognitive Learning Psychological Classification and Neural Network Design Elements

Contact Info

Product

Resources

About