Development and Validation of a Universal Science Writing Rubric That is Applicable to Diverse Genres of Science Writing

Pisano, Alycia; Crawford, A.; Huffman, Heather; Graham, Barbara; Kelp, Nicole

doi:10.1128/jmbe.00189-21

Cited by 6 publications

(8 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the history and philosophy course, which was launched recently and has a lower enrollment, fewer assignments were available, so the assignments were selected because they covered the selected topics, and a random sample was chosen from those assignments. Gound zero for the comparison was grading by one of the instructors according to rubrics he had created (Stevens and Levi, 2012;Pisano et al, 2021). The grading was made blind by having one of the authors provide writing assignments to the instructor without grade information.…”

Section: Data From the Astronomy Moocsmentioning

confidence: 99%

Using Large Language Models for Automated Grading of Student Writing about Science

Impey,

Wenger,

Garuda

et al. 2024

Preprint

View full text Add to dashboard Cite

A challenge in teaching large classes for formal or informal learners is assessing writing. As a result, most large classes, especially in science, use objective assessment tools like multiple choice quizzes. The rapid maturation of AI has created the possibility of using large language models (LLMs) to assess student writing. An experiment was carried out using GPT-3.5 and GPT-4 to see if machine learning methods based on LLMs can rival peer grading for reliability and automation in evaluating short writing assignments on topics in astronomy. The audience was lifelong learners in three massive open online courses (MOOCs) offered through Coursera. However, the results should also be applicable to non-science majors in university settings. The data was answers from 120 students on 12 questions across the three courses. The LLM was fed with total grades, model answers, and rubrics from an instructor for all three questions. In addition to seeing how reliably the LLMs reproduced instructor grades, the LLMs were asked to generate their own rubrics. Overall, the LLMs were more reliable than peer grading, both in the aggregate and by individual student, and they came much closer to the instructor grades for all three of the online courses. GPT-4 generally outperformed GPT-3.5. The implication is that LLMs can be used for automated, reliable, and scalable grading of student science writing.

show abstract

Section: Data From the Astronomy Moocsmentioning

confidence: 99%

Using Large Language Models for Automated Grading of Student Writing about Science

Impey,

Wenger,

Garuda

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Related to the discipline of science writing, the recently published USWR was designed to assess any genre of science writing, from lab reports to review articles to news reports, regardless of audience (Pisano et al, 2021). This rubric allows assessment of science writing skills across different STEM writing assignments.…”

Section: Rationale For the Assignmentmentioning

confidence: 99%

“…Students were encouraged to think about values of each audience, why they do or should care about the issue, how the audiences could be involved in solving the issue, or how the audience might benefit from a proposed solution to the issue. Students then utilized a modified version of the Universal Science Writing Rubric (USWR) (Pisano et al, 2021) to perform a peer review of each other's writing before completing final drafts of their paragraphs in response to this peer feedback.…”

mentioning

confidence: 99%

“…These diverse rhetorical concerns are all important for clear science writing to achieve its purpose. This rubric can be used for instructor grading, peer review, programmatic assessment, and more (Pisano et al, 2021). In our second iteration of this assignment, we added some more specific questions to guide student peer review based on this specific assignment.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Inclusive Science Writing about Socioscientific Issues for Diverse Audiences

Kelp

Pisano

Alderfer

et al. 2023

Prompt

View full text Add to dashboard Cite

In this paper, we present a science writing assignment in which students focus on targeting specific audiences when writing about a socioscientific issue as well as participate in a peer review process. This assignment helps students consider inclusive science communication in their writing, focusing on engaging unique audiences about the intersections of science and social justice. Students are introduced to evidence-based tools for formulating communication for unique audiences as well as for assessment of writing quality. This assignment is novel in that it helps students think about inclusion issues in STEM, science writing, and peer review, all of which are key disciplinary skills that are not always included in STEM courses. While this assignment was piloted in chemistry and environmental engineering courses, this assignment could easily be modified for other disciplines.

show abstract

“…This is why in the writing process itself, the pre-writing stage is a crucial stage for writers to prepare for what they are writing [15]. The use of a stimulus or the preparation of a writing framework, for example, is an example of a treatment that has been widely studied for its usefulness in learning to write [16]. For this reason, this finding is evidence that learning to write itself needs to be better organized based on the difficulties experienced by students as novice writers.…”

Section: "In the Introduction The Frequent Revision Is On The Backgro...mentioning

confidence: 99%

Analysis of Student’s Scientific Writing Skills in Research: Common Mistakes and Writing Style

Rini¹,

Nuraini²,

Cholifah³

2023

Proceedings of the International Conference on Educational Management and Technology (ICEMT 2022)

View full text Add to dashboard Cite

This study aims to describe the ability to write scientific papers of students from common mistakes and writing styles. Scientific writings intended in this study are in the form of a research report based on pre-research findings during student final project supervision which shows many writing problems. The research method used is in the form of content analysis by collecting data in a documentary manner and through questionnaires. Data were collected from the responses of 30 students and their research reports. Data were processed descriptively qualitatively and quantitatively to present data analysis based on content, structure, grammar, and neatness of writing. The results show that the introduction or background and discussion are the parts with the most errors and are considered difficult for students. In addition, from the writing style, it was also found that there were similarities in writing errors such as introductory topics about education, lack of coherence, and less varied word choices. Overall these findings indicate that the ability of students to write scientific papers for research reports is still low so follow-up efforts and optimization solutions are needed through ideal writing learning.

show abstract

Development and Validation of a Universal Science Writing Rubric That is Applicable to Diverse Genres of Science Writing

Abstract: It is critical for science, technology, engineering, and mathematics (STEM) students to develop competencies in science communication, including science writing. However, it can be difficult for instructors and departments to assess the quality of their students’ science writing.

Cited by 6 publications

References 20 publications

Using Large Language Models for Automated Grading of Student Writing about Science

Using Large Language Models for Automated Grading of Student Writing about Science

Inclusive Science Writing about Socioscientific Issues for Diverse Audiences

Analysis of Student’s Scientific Writing Skills in Research: Common Mistakes and Writing Style

Contact Info

Product

Resources

About