Applying Argumentation Schemes for Essay Scoring

Yi, Song; Heilman, Michael; Klebanov, Beata Beigman; Deane, Paul

doi:10.3115/v1/w14-2110

Cited by 61 publications

(69 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also compared our best across-topics system (dr pn+1-3gr ppos) to the system described in Song et al (2014). The Song et al (2014) system uses the following features: length of the sentence, parts of speech, overlap of words in the sentence with the prompt, relative position of the sentence in the essay, 1-3gr, and 1-3gr in previous and next sentences.…”

Section: Benchmarkmentioning

confidence: 99%

“…The training sets per prompt contain on average 2,700 sentences, of which 38% are classified as containing good argument critiques. Two of the three same-topic sets were used previously in Song et al (2014).…”

Section: Same-topicmentioning

confidence: 99%

“…We used Song et al (2014) annotation protocol, adapting as needed to cover additional argumentation schemes. Song et al (2017) provides a detailed exposition of the argumentation-schemebased analysis of a number of prompts and of the annotation process.…”

Section: Annotationmentioning

confidence: 99%

“…They also showed that automated acrosstopics prediction is very hard; for some topics, no effective prediction was achieved. Song et al (2014) developed an annotation protocol for analyzing argument critiques in students' essays, drawing on the theory of argumentation schemes (Walton et al, 2008;Walton, 1996). According to this theory, different types of arguments invite specific types of critiques.…”

Section: Introduction and Related Workmentioning

confidence: 99%

“…According to this theory, different types of arguments invite specific types of critiques. For example, an argument from authority made in the promptAccording to X, Y is the case -avails critiques along the lines of whether X has the necessary knowledge and is an unbiased source of information about Y. Analyzing prompts used in an assessment of argument critique skills, Song et al (2014) identified a number of common schemes, such as arguments from policy, sample, example, and used the argumentation schemes theory to specify what critiques would count as "good" for arguments from the given scheme. Once a prompt is associated with a specific set of argumentation schemes, it follows that those critiques that count as good under one of the schemes used in the prompt would be considered as good critiques in essays responding to that prompt.…”

Section: Introduction and Related Workmentioning

confidence: 99%

See 4 more Smart Citations

Detecting Good Arguments in a Non-Topic-Specific Way: An Oxymoron?

Klebanov

Gyawali

2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 2: Short Papers)

Self Cite

View full text Add to dashboard Cite

Automatic identification of good arguments on a controversial topic has applications in civics and education, to name a few. While in the civics context it might be acceptable to create separate models for each topic, in the context of scoring of students' writing there is a preference for a single model that applies to all responses. Given that good arguments for one topic are likely to be irrelevant for another, is a single model for detecting good arguments a contradiction in terms? We investigate the extent to which it is possible to close the performance gap between topicspecific and across-topics models for identification of good arguments.

show abstract

Section: Benchmarkmentioning

confidence: 99%

Section: Same-topicmentioning

confidence: 99%

Section: Annotationmentioning

confidence: 99%

Section: Introduction and Related Workmentioning

confidence: 99%

Section: Introduction and Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Detecting Good Arguments in a Non-Topic-Specific Way: An Oxymoron?

Klebanov

Gyawali

2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 2: Short Papers)

Self Cite

View full text Add to dashboard Cite

show abstract

The representation of argumentation in scientific papers: A comparative analysis of two research areas

Wang

Song

Zhou

et al. 2021

Asso for Info Science & Tech

View full text Add to dashboard Cite

Scientific papers are essential manifestations of evolving scientific knowledge, and arguments are an important avenue to communicate research results. This study aims to understand how the argumentation process is represented in scientific papers, which is important for knowledge representation, discovery, and retrieval. First, based on fundamental argument theory and scientific discourse ontologies, a coding schema, including 17 categories was constructed. Thereafter, annotation experiments were conducted with 40 scientific articles randomly selected from two different research areas (library and information science and biomedical sciences). Statistical analysis and the sequential pattern mining method were then employed; the ratio of different argumentation units and evidence types were calculated, the argumentation semantics of figures and tables analyzed, and the argumentation structures extracted. A correlation analysis between argumentation and rhetorical structures was also performed to further reveal how argumentation was represented within scientific discourses. The results indicated a difference in the proportion of the argumentation units in the two types of scientific papers, as well as a similar linear construction with differences in the specific argument structures of each knowledge domain and a clear correlation between argumentation and rhetorical structure.Scientific papers are the most essential manifestations of evolving scientific knowledge, and argumentation is an important avenue through which research results are communicated (Walton & Zhang, 2013). Scientific discovery challenges existing theoretical analyses and empirical studies in a process of continuous argumentation; thus, argumentation forms the foundation of scientific papers, and articles demonstrating a rigorous argumentation process are essential for scientific development. Meanwhile, the "argumentation units" (along with the argumentation structure within a scientific paper) are a special implicit knowledge entity that represents the attribute of knowledge.Argumentation plays a critical role in identifying relevant arguments based on their content; thus, it seems reasonable to focus on identifying characteristic patterns of argumentation and the ways in which they are typically developed (Song et al., 2014). Revealing, extracting, and analyzing the argumentation structure of scientific papers plays an important role in interpreting the general rules of their content composition and benefits knowledge discovery, representation, retrieval, and ultimately access. Therefore, it is necessary to understand how argumentation is represented within scientific papers and to identify argument structures when they are extended to whole text segments (Walton, 2013;Walton et al., 2008).Recent research has placed particular emphasis on the definition, description, and representation of scientific content from the semantic and pragmatic perspectives. Several ontologies, such as the document element

show abstract

Toward the Automated Scoring of Written Arguments: Developing an Innovative Approach for Annotation

Deane

Klebanov

2017

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

This project focuses on laying the foundations for automated analysis of argumentation schemes, supporting identification and classification of the arguments being made in a text, for the purpose of scoring the quality of written analyses of arguments. We developed annotation protocols for 20 argument prompts from a college‐level test under the framework of the theory of argumentation schemes, which defines reasoning patterns in argumentation. The annotation protocols listed critical questions associated with each argumentation scheme, which makes the argument structure in a text explicit and classifiable. Furthermore, we annotated 200 student essays across four selected argument prompts to test if the annotation protocols can be applied reliably by human annotators. Preliminary results indicate that this method of analyzing argument structure is reliable and promising.

show abstract

Applying Argumentation Schemes for Essay Scoring

Cited by 61 publications

References 18 publications

Detecting Good Arguments in a Non-Topic-Specific Way: An Oxymoron?

Detecting Good Arguments in a Non-Topic-Specific Way: An Oxymoron?

The representation of argumentation in scientific papers: A comparative analysis of two research areas

Toward the Automated Scoring of Written Arguments: Developing an Innovative Approach for Annotation

Contact Info

Product

Resources

About