Evaluating Neural Networks as a Method for Identifying Students in Need of Assistance

Abstract

Course instructors need to be able to students in need of assistance as early in the course as possible. Recent work has suggested that machine learning approaches applied to snapshots of small programming exercises may be an effective solution to this problem. However, these results have been obtained using data from a single institution, and prior work using features extracted from student code has been highly sensitive to differences in context. This work provides two contributions: first, a partial reproduction of previously published results, but in a different context, and second, an exploration of the efficacy of neural networks in solving this problem. Our findings confirm the importance of two features (the number of steps required to solve a problem and the correctness of key problems), indicate that machine learning techniques are relatively stable across contexts (both across terms in a single course and across courses), and suggest that neural network based approaches are as effective as the best Bayesian and decision tree methods. Furthermore, neural networks can be tuned to be reliably pessimistic, so they may serve a complementary role in solving the problem of identifying students who need assistance.

Resilience and Effective Learning in First-Year Undergraduate Computer Science

“…Recently, artificial intelligence and machine learning approaches -especially in the context of a wider higher education push on learning analytics -have been widely applied in success / retention prediction (e.g. [1,7,22,28]). However, there is limited published work related to the prediction of a computer science student's overall results and attendance based on measures of positive psychology, which is the focus of this work.…”

“…In the domain of CS, threshold concepts have be argued to be largely related to programming [34], leading to research related to success seen through the lens of programming (commonly fundamental programming or "CS1"). This has been productive but can been argued to leave a gap in our understanding of factors that can be predictors of success [7,22,28]. In particular, in the case of troublesome knowledge, learners may need to perform in the face of adversity.…”

Um Mapeamento Sistemático sobre Métodos de Identificação Preditiva de Alunos com Risco de Reprovação em Educação de Computação

“…Em [Ahadi 2016], utilizou-se uma ferramenta capaz de capturar snapshots dos códigos-fonte produzidos pelos alunos, em um curso introdutório de programação, de modo que o algoritmo identificava padrões de alunos com dificuldades. Já em [Castro-Wunsch et al 2017], com o auxílio de uma Rede Neural Artificial (RNA), foi realizada a identificação dos alunos que estão em risco de reprovação. Uma outra abordagem utilizava uma ferramenta que coletava o número de submissões feitas e a quantidade de dicas usadas pelos alunos para que o modelo criado pudesse realizar o cálculo das Métricas de Trajetória e de Linha de Base para a identificação dos alunos que necessitavam de auxílio [Estey et al 2017].…”

“…De forma semelhante, em [Ahadi 2016], foi realizada a coleta os dados com uma ferramenta capaz de capturar snapshots dos códigos-fonte gerados, podendo apresentar ao professor as dificuldades dos alunos. Em [Castro-Wunsch et al 2017], a partir de dados obtidos e processados por uma RNA, obteve-se um bom resultado na predição do sucesso dos alunos, com resultado de 80-85% de variação em sua eficiência. A ferramenta apresentada em [Estey et al 2017] tem o diferencial de retornar um feedback para o aluno, de forma que, além do professor, o aluno também pode saber como está sua compreensão em relação aos conteúdos.…”

Optimization of Classifiers Ensemble Construction: Case Study of Educational Data Mining

“…Educational Data Mining and Learning Analytics (EDM/LM) are promising scientific field to enhance of teaching and learning technologies of traditional and e-learning education [1][2][3][4][5] and to manage of various forms of constructivist education [6]. The wide availability of data mining tools such as R, scikit-learn for Pyton, and Weka [7] allows us to solve one of the main tasks of EDM/LA: to forecast of student's performance and to help the needy [8,9]. Most commonly, this task is resolved by using of individual classifiers with learning following algorithms [10]: Naïve Bayes (NB), Decision Tree (J48), Multi-Layer Perceptron (MLP), Nearest Neighbors (1NN) and Support Vector Machine (SVM) and other algorithms from the top-10 list [11].…”