Automated essay scoring and the future of educational assessment in medical education

Gierl, Mark J.; Latifi, Syed; Lai, Hollis; Boulais, André-Philippe; Champlain, André De

doi:10.1111/medu.12517

Cited by 48 publications

(33 citation statements)

References 20 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Depending upon the underlying technology of AES and question type, the typical sample size estimates needed range from 100 to 1,000 prescored essays (Dikli, 2006). This outcome implies that AES may be impractical in small assessment settings (Gierl et al, 2014). However, the typical examinee cohort of MCC Part I Qualifying Examination is well above the recommended sample size guidelines, thereby making this scoring framework more practical in such settings.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

“…AES offers many benefits for scoring written assessments, such as improving the consistency of scoring, reducing the time required for scoring and reporting, minimizing the costs of scoring, and providing students with immediate feedback on their written responses (Weigle, 2013;Williamson, 2013). A comprehensive review on the application of AES in medical education was recently published by Gierl, Latifi, Lai, Boulais, and De Champlain (2014).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Using Automated Scoring to Evaluate Written Responses in English and French on a High-Stakes Clinical Competency Examination

et al. 2015

Self Cite

View full text Add to dashboard Cite

We present a framework for technology-enhanced scoring of bilingual clinical decision-making (CDM) questions using an open-source scoring technology and evaluate the strength of the proposed framework using operational data from the Medical Council of Canada Qualifying Examination. Candidates' responses from six write-in CDM questions were used to develop a three-stage-automated scoring framework. In Stage 1, the linguistic features from CDM responses were extracted. In Stage 2, supervised

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Using Automated Scoring to Evaluate Written Responses in English and French on a High-Stakes Clinical Competency Examination

et al. 2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…Especially in high-stakes exams, such as in the medical field, consistency and fairness in the testing system need to be emphasized, and using machine learning to grade exams can reduce uncertainty. Other ramifications include saving time and money by reducing the need for human markers, as well as the opportunity of giving immediate feedback to students (Gierl et al, 2014). With powerful algorithms ready to be used, it is only a matter of time before the educational system sees machines marking essays from a large variety of fields.…”

Section: Future Directionsmentioning

confidence: 99%

The Application of Machine Learning to Education

2017

JSST

View full text Add to dashboard Cite

The field of education is constantly seeking more innovative and effective methods of teaching foundational knowledge to students. Organizations in both secondary and post-secondary education groups offer Physics Education Groups that try to better teach fundamental physics concepts to students in both university and high school. There are also government agencies such as the Educational Quality and Accountability Office (EQAO) in Ontario, which provides standardized tests to students. Although these standardized tests do not test the full capabilities and thought processes of students, they still provide insights into how students learn. Data from standardized EQAO tests can be analyzed to obtain crucial information about how to improve education standards by showing where resources can be allocated more effectively. One of the most powerful tools that can be used to analyze data is machine learning, which can find patterns and correlations in data that the human eye cannot see. This experiment used the linear regression algorithm to find correlations in data obtained from grade 9 EQAO mathematics tests from 50 schools in Ontario. The algorithm analyzed how students with varying scores answered a multiple-choice questionnaire at the end of the exam, which included statements such as "I am able to answer difficult mathematics questions." Based on the variables output from the machine learning algorithm, the importance of each statement was then ranked; this ranking can then lead to insights into how students learn, and how to better utilize resources. This experiment has shown that an elementary application of machine learning can lead to valuable insights into student learning and that more should be done to better analyze the abundant data in the education field.Le domaine de l'éducation recherche continuellement des méthodes innovatrices et efficaces pour l'enseignement de connaissances fondamentales aux étudiants. Les organisations de niveau secondaire et post-secondaire offrent des groupes d'éducation en physique qui essaient d'améliorer l'enseignement des concepts fondamentaux en physique aux étudiants à l'université ainsi qu'au secondaire. Il y a également des agences gouvernementales telles que l'Office de la qualité et de la responsabilité en éducation (OQRE) en Ontario, qui offre des tests standardisés aux étudiants. Bien que ces tests standardisés n'évaluent pas la pleine capacité et le processus de réflexion des élèves, ils offrent tout de même un aperçu des méthodes d'apprentissage. Les données des tests standardisés de l'OQRE peuvent être analysés afin d'obtenir des renseignements essentiels quant à la façon d'améliorer les normes d'éducation en démontrant où les ressources peuvent être attribuées plus efficacement. Un des outils les plus puissants qui peut être utilisé pour analyser les données est le l'apprentissage automatique, ce qui peut trouver des tendances et des corrélations à travers les données que l'oeil humain ne peut percevoir. Cette expérience a utilisé l'algorithme à régress...

show abstract

“…Robotic drug‐dispensing machines are replacing pharmacists just as bank machines replaced tellers. In education, testing organisations are poised to augment human test committees with computers that generate test items and score examinations …”

mentioning

confidence: 99%

Learning from Dorothy Vaughan: artificial intelligence and the health professions

Hodges¹

2017

Med Educ

View full text Add to dashboard Cite

show abstract

Automated essay scoring and the future of educational assessment in medical education

Cited by 48 publications

References 20 publications

Using Automated Scoring to Evaluate Written Responses in English and French on a High-Stakes Clinical Competency Examination

Using Automated Scoring to Evaluate Written Responses in English and French on a High-Stakes Clinical Competency Examination

The Application of Machine Learning to Education

Learning from Dorothy Vaughan: artificial intelligence and the health professions

Contact Info

Product

Resources

About