Abstract:Genre serves as a useful lens to investigate the range of evidence derived from automated writing evaluation (AWE). To support construct-relevant systems used for writing instruction and assessment, two investigations were conducted that focused on post-secondary writing requirements and faculty perceptions of student writing proficiency. Survey research is described from a national study and a second site study at American University, a 4-year private university in Washington, DC, to illustrate writing requir… Show more
“…Drawing on recent work that looked at faculty perceptions related to students' weak writing quality at 4-year post- Figure 1. Claims activity: The student highlights language indicating presence of claims; the system highlights additional terms indicating presence of claims secondary institutions, the demo mock-up is organized into writing quality categories [4], [19]. It is intended to provide guidance and encourage students to closely review and revise critical writing construct elements aligned with a variety of writing genres across academic disciplines.…”
Section: Demo Designmentioning
confidence: 99%
“…Weak written communication may be a contributing factor. A disconnect has persisted for writing requirements across the K-12 and postsecondary trajectory [4], [5], [6]. Recent research has exposed a set of writing domain knowledge skills [7] for which university faculty perceived student deficiency [4].…”
Writing is key to educational and workplace success. The majority of computer-based writing support applications target K-12 students and offer standard feedback concerning the U.S.centric, 5-paragraph, expository essay genre, which is typically taught to developing writers and used on standardized tests. This writing demo takes a "left turn". By "left turn", we mean that this demo is illustrating a new writing support concept. In contrast to most systems that provide only feedback, this demo illustrates how guided activities can be automatically generated as a complement to relevant writing feedback driven by natural language processing (NLP) methods. Further, the demo is intended to handle a wide range of writing genres in postsecondary academic settings. The demo feedback and activities illustrated in this paper are ideas that are intended to promote postsecondary students' engagement with writing assignments, and address writing domain knowledge that is critical for producing higher quality postsecondary writing assignments across writing genres and academic disciplines.
“…Drawing on recent work that looked at faculty perceptions related to students' weak writing quality at 4-year post- Figure 1. Claims activity: The student highlights language indicating presence of claims; the system highlights additional terms indicating presence of claims secondary institutions, the demo mock-up is organized into writing quality categories [4], [19]. It is intended to provide guidance and encourage students to closely review and revise critical writing construct elements aligned with a variety of writing genres across academic disciplines.…”
Section: Demo Designmentioning
confidence: 99%
“…Weak written communication may be a contributing factor. A disconnect has persisted for writing requirements across the K-12 and postsecondary trajectory [4], [5], [6]. Recent research has exposed a set of writing domain knowledge skills [7] for which university faculty perceived student deficiency [4].…”
Writing is key to educational and workplace success. The majority of computer-based writing support applications target K-12 students and offer standard feedback concerning the U.S.centric, 5-paragraph, expository essay genre, which is typically taught to developing writers and used on standardized tests. This writing demo takes a "left turn". By "left turn", we mean that this demo is illustrating a new writing support concept. In contrast to most systems that provide only feedback, this demo illustrates how guided activities can be automatically generated as a complement to relevant writing feedback driven by natural language processing (NLP) methods. Further, the demo is intended to handle a wide range of writing genres in postsecondary academic settings. The demo feedback and activities illustrated in this paper are ideas that are intended to promote postsecondary students' engagement with writing assignments, and address writing domain knowledge that is critical for producing higher quality postsecondary writing assignments across writing genres and academic disciplines.
“…Evidence suggests that students have difficulty with genres of writing that include citing and integrating sources, such as annotated bibliographies and literature reviews, which are commonly required assignments in college courses (Burstein, Elliot, & Molloy, in press). Understanding the elements of these assignments that pose difficulty for students can give some indication of the types of skills and prerequisites that might be addressed in K–12 education, as emphasis shifts toward an increasing focus on engaging students in research from multiple sources (Coiro & Kennedy, ).…”
Current educational standards call for students to engage in the skills of research and inquiry, with a focus on gathering evidence from multiple information sources, evaluating the credibility of those sources, and writing an integrated synthesis that cites evidence from those sources. Opportunities to build strong research skills are critical, yet empirical research demonstrates that students from Grades K–16 struggle with inquiry tasks, particularly in online environments. There is a need to create models that will support teachers in developing students' research skills and can be used to develop reliable and valid assessments of such skills while aligning with standards. Under the CBAL™ research initiative, we have developed a model of conducting research and inquiry as a key literacy practice in the English language arts (ELA). In this paper, we draw on literature from the cognitive and learning sciences—including work in discourse processing, science education, educational technology, and information literacy—to provide the theoretical background for this key practice. We identify a set of activities and skills that are critical for participating in research; each skill is accompanied by a set of provisional learning progressions, which outlines tentative predictions about the qualitative changes in a skill that develop over time with appropriate instruction. These learning progressions and their relation to the key practice can be leveraged in the design of cognitively based assessments of research and inquiry that are sensitive to students' developmental level. We conclude, with an example design for such an assessment to illustrate how key practices and learning progressions can be integrated to support measurement of research and inquiry skills.
“…Genres of writing, for example, serve specific social goals and purposes (Bazerman, ), and those rhetorical goals shape and constrain the types and methods through which information should be recorded and shared with others when writing within a particular genre. In higher education, the focus is typically on transactional writing (i.e., writing to communicate or exchange information, ideas, or arguments with others in order to achieve particular purposes, such as to inform, persuade, or explain information to others; Burstein et al, ). Therefore, writers must consider the nature and needs of their audience(s) in order for communication to be successful (cf.…”
Section: Review Of Existing Framework and Researchmentioning
confidence: 99%
“…Authenticity can be defined as the extent to which the features of an assessment task correspond to the features of the situations in which the skills being assessed will be used and applied in the real world (Bachman & Palmer, ). For higher education, the notion of authentic writing assessment suggests that the tasks included in the assessment design should correspond to the types of writing assignments required of students in their undergraduate coursework, such as writing arguments or research articles (Burstein et al, ). The notion of authentic writing assessment is consistent with the position statement on writing assessment released by the Conference on College Composition and Communication (CCCC, ), which asserts that best assessment practices ask students to produce writing within a meaningful context: The assessment of writing must strive to set up writing tasks and situations that identify purposes appropriate to and appealing to the particular students being tested… .…”
Section: Review Of Existing Assessments and Design Challengesmentioning
Written communication is considered one of the most critical competencies for academic and career success, as evident in surveys of stakeholders from higher education and the workforce. Emphasis on writing skills suggests the need for next‐generation assessments of writing proficiency to inform curricular and instructional improvement. This article presents a comprehensive review of definitions of writing proficiency from key higher education and workforce frameworks; the strengths and weaknesses of existing assessments; and challenges related to designing, implementing, and interpreting such assessments. Consistent with extant frameworks, we propose an operational definition including 4 strands of skills: (a) social and rhetorical knowledge, (b) domain knowledge and conceptual strategies, (c) language use and conventions, and (d) the writing process. Measuring these aspects of writing requires multiple assessment formats (including selected‐response [SR] and constructed‐response [CR] tasks) to balance construct coverage and test reliability. Next‐generation assessments should balance authenticity (e.g., realistic writing tasks) and psychometric quality (e.g., desirable measurement properties), while providing institutions and faculty with actionable data. The review and operational definition presented here should serve as an important resource for institutions that seek to either adopt or design an assessment of students' writing proficiency.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
