In the tradition of Swalesian genre theory, this manuscript explores the rhetorical composition of research article Methods sections through a top-down analysis of a corpus of nine hundred texts representative of thirty academic fields. The analysis resulted in a comprehensive cross-disciplinary model, called Demonstrating Rigour and Credibility (DRaC). The model contains three moves and sixteen steps, which are defined in terms of functional and content realizations. DRaC further served as the analytic framework for corpus annotation. Manually annotated corpus data revealed the moves and steps with high distributional prominence as well as those that are not frequent but occur consistently within and across disciplines. Visualizations of individual texts in a sample of disciplines demonstrated inter-disciplinary and intradisciplinary patterns and variation in move sequencing. Additionally, algorithmic analysis of the annotated corpus showed that soft and hard sciences form clusters based on their use of DRaC steps, providing a deeper understanding of how shared conventions of rhetorical composition distinguish cross-disciplinary similarities in Methods discourse. The findings lend themselves to application in genre writing pedagogy and, more broadly, hold implications for theories of social and cognitive genres.

Two examples demonstrate an argument-based approach to validation of diagnostic assessment using automated writing evaluation (AWE). Criterion ® , was developed by Educational Testing Service to analyze students' papers grammatically, providing sentence-level error feedback. An interpretive argument was developed for its use as part of the diagnostic assessment process in undergraduate university English for academic purposes (EAP) classes. The Intelligent Academic Discourse Evaluator (IADE) was developed for use in graduate EAP university classes, where the goal was to help students improve their discipline-specific writing. The validation for each was designed to support claims about the intended purposes of the assessments. We present the interpretive argument for each and show some of the data that have been gathered as backing for the respective validity arguments, which include the range of inferences that one would make in claiming validity of the interpretations, uses, and consequences of diagnostic AWE-based assessments. New diagnostic assessments of writing with automated writing evaluation (AWE) promise individualized diagnostic feedback aimed at guiding students' revision and raising their consciousness about specific aspects of their writing to help them learn. However, like any assessment, diagnostic assessments need to be evaluated in view of the validity of their intended interpretations, uses and consequences. Despite the need for validity arguments, the evaluation of AWE has tended to focus on accuracy of the system (Dikli, 2006), even if other approaches to validation have been recognized (Bennett and Bejar, 1998; Yang, Buckendahl, Juszkiewicz, & Bhola, 2002;Cotos & Pendar, 2008;Powers, Burstein, Chodorow, Fowles, & Kukich, 2001; Yang et al. 2002). In particular, Clauser, Kane and Swanson (2002) conceptualized how to frame a validity argument by identifying five types of inferences that one might wish to make on the basis of results from an assessment with automated scoring. This paper demonstrates how such inferences can be used to develop validity arguments for two AWE-based diagnostic assessments. This paper illustrates 1) the claims that users of AWE systems want to be able to make about the value of the classroom assessment and the data used to support such claims, as well as 2) the need for inferences and claims to be developed to reflect the goals of the AWE system and its use. We provide two examples to demonstrate that the variety of approaches taken to AWE-based diagnostic assessment can be evaluated through the use of an argument-based validity framework. The first system, Criterion®, was developed by Educational Testing Service to analyze students' papers grammatically and provide sentence-level error feedback. This system was used as part of the diagnostic assessment in five undergraduate university English for academic purposes (EAP) classes. The second one, Intelligent Academic Discourse Evaluator (IADE), was developed for use in six graduate EAP university classes, wher...

This paper presents an empirical evaluation of automated writing evaluation (AWE) feedback used for L2 academic writing teaching and learning. It introduces the Intelligent Academic Discourse Evaluator (IADE), a new web-based AWE program that analyzes the introduction section to research articles and generates immediate, individualized, and discipline-specific feedback. The purpose of the study was to investigate the potential of IADE's feedback. A mixed-methods approach with a concurrent transformative strategy was employed. Quantitative data consisted of responses to Likert-scale, yes/no, and open-ended survey questions; automated and human scores for first and final drafts; and pre-/posttest scores. Qualitative data contained students' first and final drafts as well as transcripts of think-aloud protocols and Camtasia computer screen recordings, observations, and semistructured interviews. The findings indicate that IADE's colorcoded and numerical feedback possesses potential for facilitating language learning, a claim supported by evidence of focus on discourse form, noticing of negative evidence, improved rhetorical quality of writing, and increased learning gains.

Learner corpora have become prominent in language teaching and learning, enhancing data-driven learning (DDL) pedagogy by promoting 'learning driven data' in the classroom. This study explores the potential of a local learner corpus by investigating the effects of two types of DDL activities, one relying on a native-speaker corpus (NSC) and the second combining native-speaker and learner corpora. Both types of activities aimed at improving second language writers' knowledge of linking adverbials and were based on a preliminary analysis of adverbial use in the local learner corpus produced by 31 study participants. Quantitative and qualitative data, obtained from writing samples, pre/post-tests, and questionnaires, were converged through concurrent triangulation. The results showed an increase in frequency, diversity and accuracy in all participants' use of adverbials, but more significant improvement was made by the students who were exposed to the corpus containing their own writing. The findings of this study are thus interpreted as suggestive that combining learner and native-speaker data is a feasible and effective practice, which can be readily integrated in DDL-based instruction with positive impact.

John Swales' seminal work has inspired a wealth of research with important pedagogical implications for genre-based writing instruction. Continuing the prolific move analysis tradition in EAP research, this article presents empirically devised and validated cross-disciplinary IMRD move/step frameworks for the research article genre and demonstrates how Swales' move and step concepts underlying these frameworks formed the foundation of innovative genre-based automated writing evaluation technology. Overall, this paper makes the relationships between genre theory, genre analysis, and genre instruction explicit, demonstrating that move analysis is a powerful and promising theoretical, analytic, and teaching construct. With that, we take Swales' vision to a new dimension of conceptualizing EAP. Keywordsgenre analysis, move, step, genre-based writing pedagogy, automated writing evaluation, research writing Disciplines Curriculum and Instruction | Curriculum and Social Inquiry | Educational Assessment, Evaluation, and Research | Educational MethodsComments NOTICE: this is the author's version of a work that was accepted for publication in Journal of English for Academic Purposes. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Journal of English for Academic Purposes, [19, (2015) AbstractJohn Swales' seminal work has inspired a wealth of research with important pedagogical implications for genre-based writing instruction. Continuing the prolific move analysis tradition in EAP research, this article presents empirically devised and validated cross-disciplinary IMRD move/step frameworks for the research article genre and demonstrates how Swales' move and step concepts underlying these frameworks formed the foundation of innovative genre-based automated writing evaluation technology. Overall, this paper makes the relationships between genre theory, genre analysis, and genre instruction explicit, demonstrating that move analysis is a powerful and promising theoretical, analytic, and teaching construct. With that, we take Swales' vision to a new dimension of conceptualizing EAP.