iSTART: Adaptive Comprehension Strategy Training and Stealth Literacy Assessment

McNamara, Danielle S.; Arner, Tracy; Butterfuss, Reese; Ying, Fang; Watanabe, Micah; Newton, Natalie; McCarthy, Kathryn S.; Roscoe, Rod D.

doi:10.1080/10447318.2022.2114143

Cited by 7 publications

(3 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this system, the AI agent prompts students to brainstorm potential solutions and then uses scaffolding to guide them toward solving problems more effectively. Other studies in the area of language comprehension have also suggested the effectiveness of AI-assisted questioning and feedback in improving students' comprehension (K. S. McCarthy et al, 2020;McNamara et al, 2023).…”

Section: Ai For Personalization Of Learningmentioning

confidence: 99%

Artificial intelligence enhances children’s science learning from television shows.

Xu,

He,

Levine

et al. 2024

Journal of Educational Psychology

View full text Add to dashboard Cite

Educational television programs are important learning resources for young children, especially those from underresourced households. These programs’ potential can be amplified if children are given the opportunity to meaningfully interact with media characters during their video watching. In this project, we partnered with PBS KIDS to develop interactive science-focused videos in which the main character, powered by artificial intelligence, engaged in dialogic interactions with children by asking them questions and providing responsive feedback. The children who watched the interactive videos performed better on a science posttest than children who watched the broadcast version of the video (without any interaction) or a pseudointeractive version (in which the media character asks children the same questions and gives generic feedback after a fixed amount of time). The artificial intelligence character’s responsiveness positively influenced both the quantity and quality of children’s verbalizations during video watching, compared to the pseudointeractive condition. This article sheds light on the feasibility and effectiveness of using conversational technologies to support active learning in children through educational programs.

show abstract

Section: Ai For Personalization Of Learningmentioning

confidence: 99%

Artificial intelligence enhances children’s science learning from television shows.

Xu,

He,

Levine

et al. 2024

Journal of Educational Psychology

View full text Add to dashboard Cite

show abstract

“…Students are not aware that they are being assessed, which is less stressful and allows them to engage in an authentic learning task. Developing research demonstrates use of stealth assessment to evaluate students' proficiencies, such as their general vocabulary knowledge or their performance on a general reading comprehension test (McNamara et al, 2023).…”

Section: Personalized Instructionmentioning

confidence: 99%

Reading Comprehension and Constructive Learning: Policy Considerations in the Age of Artificial Intelligence

McCarthy,

Yan

2023

Policy Insights from the Behavioral and Brain Sciences

Self Cite

View full text Add to dashboard Cite

Although reading in the digital age looks different than in the past (e.g., on a screen, shorter texts spread across multiple sources), reading continues to be a central part of everyday life. Research in reading comprehension shows that this type of modern reading requires more complex knowledge building inference processes that are difficult for many adolescents and adults. Constructive and personalized learning activities can support readers’ knowledge building. Researchers have been using artificial intelligence (AI) to make these types of activities more effective and accessible. Emerging directions and considerations also result from the introduction of generative AI. Increased collaboration across researchers, developers, educators, and policymakers would afford empirically supported research, development, and implementation that can keep pace with the quickly evolving technological landscape.

show abstract

“…However, additional benefits can be realized when the reader receives feedback about the accuracy or quality of his/her self-explanation [3]. One way readers can receive feedback is from instructors who review and score self-explanations based on a rubric [4]. This method is time-consuming and does not provide readers with the feedback they need in real-time.…”

Section: Introductionmentioning

confidence: 99%

Automated Assessment of Comprehension Strategies from Self-Explanations Using LLMs

Nicula,

Dascalu,

Arner

et al. 2023

Information

Self Cite

View full text Add to dashboard Cite

Text comprehension is an essential skill in today’s information-rich world, and self-explanation practice helps students improve their understanding of complex texts. This study was centered on leveraging open-source Large Language Models (LLMs), specifically FLAN-T5, to automatically assess the comprehension strategies employed by readers while understanding Science, Technology, Engineering, and Mathematics (STEM) texts. The experiments relied on a corpus of three datasets (N = 11,833) with self-explanations annotated on 4 dimensions: 3 comprehension strategies (i.e., bridging, elaboration, and paraphrasing) and overall quality. Besides FLAN-T5, we also considered GPT3.5-turbo to establish a stronger baseline. Our experiments indicated that the performance improved with fine-tuning, having a larger LLM model, and providing examples via the prompt. Our best model considered a pretrained FLAN-T5 XXL model and obtained a weighted F1-score of 0.721, surpassing the 0.699 F1-score previously obtained using smaller models (i.e., RoBERTa).

show abstract

iSTART: Adaptive Comprehension Strategy Training and Stealth Literacy Assessment

Cited by 7 publications

References 64 publications

Artificial intelligence enhances children’s science learning from television shows.

Artificial intelligence enhances children’s science learning from television shows.

Reading Comprehension and Constructive Learning: Policy Considerations in the Age of Artificial Intelligence

Automated Assessment of Comprehension Strategies from Self-Explanations Using LLMs

Contact Info

Product

Resources

About