Automatic Assessment of Complex Assignments using Topic Models

Kuzi, Saar; Cope, Bill; Ferguson, Duncan C.; Geigle, Chase; Zhai, ChengXiang

doi:10.1145/3330430.3333615

Cited by 9 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Automated assessments help reduce instructor burden and enhance students' learning. Much of the work on automated assessment using learning algorithms has been implemented in complex domains, such as open-ended learning environments [31] and essay grading [21]. However, most of the work in CS education domain has been limited to using test-cases for assessment (e.g.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Toward Semi-Automatic Misconception Discovery Using Code Embeddings

Shi

Shah

Wang

et al. 2021

LAK21: 11th International Learning Analytics and Knowledge Conference

View full text Add to dashboard Cite

Understanding students' misconceptions is important for effective teaching and assessment. However, discovering such misconceptions manually can be time-consuming and laborious. Automated misconception discovery can address these challenges by highlighting patterns in student data, which domain experts can then inspect to identify misconceptions. In this work, we present a novel method for the semi-automated discovery of problem-specific misconceptions from students' program code in computing courses, using a state-of-the-art code classification model. We trained the model on a block-based programming dataset and used the learned embedding to cluster incorrect student submissions. We found these clusters correspond to specific misconceptions about the problem and would not have been easily discovered with existing approaches. We also discuss potential applications of our approach and how these misconceptions inform domain-specific insights into students' learning processes. CCS CONCEPTS• Computing methodologies → Learning latent representations; • Applied computing → Education.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Deep learning methods have already been used to accurately assess student performance on complex tasks (e.g. essay writing [21]). In doing so, they also learn to represent students' submissions as vectors, which makes it easier to visualize and cluster submissions and find meaningful patterns that correspond to students' performance.…”

Section: Introductionmentioning

confidence: 99%

Toward Semi-Automatic Misconception Discovery Using Code Embeddings

Shi

Shah

Wang

et al. 2021

LAK21: 11th International Learning Analytics and Knowledge Conference

View full text Add to dashboard Cite

show abstract

“…This is the case with our online writing and assessment environment, called Scholar, that since 2009 with the support of the Institute of Educational Sciences, the Gates Foundation, and sub-award funding from NSF Olmanson, et al, 2015) we have taken full advantage of new e-learning affordances to pedagogically design and develop it based on a reflexive ideology whereby complex reasoning skills such as critical and creative thinking form part of the process. Additionally, Computer Science collaborators employed available learning analytics and data mining techniques to automate critical thinking assessment through topic modelling (Zhai, 2008;Kuzi, et al, 2018), as well as semantically analyzing text to assess peer reviews (Zhai, et al, 2004;Shubhra-Kanti, et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

A ‘Digital Pedagogies’–based Learning Platform

Montebello¹

2020

Education and New Developments 2020

View full text Add to dashboard Cite

Virtual Learning Environments and Online Learning Platform have been around for some time, yet the problem with the majority of them is that they either attempt to replicate the Face-to-Face equivalent, or are simply development and deployed without a learning model or theory at their basis. In this paper we present a bottom-up development of a learning platform that is entirely based on a sound educational principles that include Ubiquitous learning, Active knowledge making, Multimodal meaning, Recursive feedback, Collaborative intelligence, Metacognition, and Differentiated learning. We will show how this platform has been employed for these last few years and has provided ample data to evaluate the added advantage that digital pedagogies provide especially as the same data is employed through the learning analytics feature that has embellished the platform in this last year. We draw numerous conclusions and recommendation on our experiences as well as on the feedback received from numerous learners and educators that have made good use of the same platform.

show abstract

“…'Big data' and 'artificial intelligence' in education end the polarizing distinctions of quantitative and qualitative research (Cope and Kalantzis 2015b). There is no meaningful quantitative research without data models (ontology); and are no qualitative meanings in the world that cannot, by transposition, be quantified, then brought back to life as qualitative meaning-hence, among other methods, natural language processing, image recognition and text mining (Kuzi et al 2019, Shubhra Kanti et al 2018, Zhai and Massung 2016.…”

mentioning

confidence: 99%

Futures for research in education

Cope¹,

Kalantzis

2020

Educational Philosophy and Theory

View full text Add to dashboard Cite

Ours are times of unprecedented 'disruption'. The business pundits like this idea. 'Disruptive innovation', they are inclined to call it (Bower and Christensen 1995), a contemporary variation of Joseph Schumpeter's old paean for capitalism as a system of 'creative destruction' (Schumpeter 1950 [1976]). The destruction fills our news feeds: the crises of the global environment; the crises of expanding inequality; the crises of legitimacy in liberal democracies; the crises of interpersonal 'isms' that reveal signs of antipathy to human differences; personal debt crises when the social goods of education, housing and health are funded by individuals who lack the means to pay; the crises of digital technologies of understanding that are beyond our understanding; and perhaps a crisis that affects all the others, a crisis of the possibility of 'truth'. These are pervasive causes for profound existential anxiety. Education as a scholarly endeavour and practical social project stands in an unusual relationship to these crises. The natural sciences can focus their efforts on averting environmental calamity. Economics can refigure distributive mechanisms. Political science can analyse the malaise in our processes and systems of government. The humanities and social sciences can address sexism, racism and other interpersonal iniquities in their historical and institutional contexts. Political economy can propose solutions that will offer more equitable access to education, housing and healthcare. Computer science can try to make its artificial intelligence explainable and accountable. But education must work on all of these things. And much more. What do we mean by 'education,' as a widely responsible intellectual practice and social project? Before beginning to address this question, this is what we don't mean. Education shouldn't merely be an exercise in narrowly 'instrumental reason' (Horkheimer 1974). The peak funding body for education research in the United States, the Institute of Educational Sciences, measures its success with research that demonstrates 'what works' (https:// ies.ed.gov/ncee/wwc/) An intervention under examination-in mathematics, literacy, whatevercan be demonstrated to produce better test results than 'business as usual'. IES is a creature of the No Child Left Behind law of 2001, a collaborative effort between George W. Bush's White House and Senator Edward Kennedy. Reviewing ten years of activity during which the Institute's annual budget was approximately $600 million annually, a report of the Government Accountability Office concluded that its 'relevance to the field is unclear' (United States Government Accountability Office 2013: 28). Perhaps the reason for this is the cumbersome and expensive quantitative 'objectivity' of the Institute's required methods; but perhaps also, in a society of chronic inequality, many children will be left behind by systemic design, and no amount of tinkering with interventions can right this wrong. Not that we haven't been grateful recipients of the Institute's g...

show abstract

Automatic Assessment of Complex Assignments using Topic Models

Cited by 9 publications

References 17 publications

Toward Semi-Automatic Misconception Discovery Using Code Embeddings

Toward Semi-Automatic Misconception Discovery Using Code Embeddings

A ‘Digital Pedagogies’–based Learning Platform

Futures for research in education

Contact Info

Product

Resources

About