The availability of the Rhetorical Structure Theory (RST) Discourse Treebank has spurred substantial research into discourse analysis of written texts; however, limited research has been conducted to date on RST annotation and parsing of spoken language, in particular, nonnative spontaneous speech. Considering that the measurement of discourse coherence is typically a key metric in human scoring rubrics for assessments of spoken language, we initiated a research effort to obtain RST annotations of a large number of non-native spoken responses from a standardized assessment of academic English proficiency. The resulting inter-annotator κ agreements on the three different levels of Span, Nuclearity, and Relation are 0.848, 0.766, and 0.653, respectively. Furthermore, a set of features was explored to evaluate the discourse structure of non-native spontaneous speech based on these annotations; the highest performing feature showed a correlation of 0.612 with scores of discourse coherence provided by expert human raters.

No abstract

We present a tool that provides automated feedback to students studying Spanish writing. The feedback is given for four categories: topic development, coherence, writing conventions, and essay organization. The tool is made freely available via a Google Docs add-on. A small user study with post-secondary level students in Mexico shows that students found the tool generally helpful and that most of them plan to continue using it as they work to improve their writing skills. In an analysis of 6 months of user data, we see that a small number of users continue to engage with the app, even outside of planned user studies.

Automated scoring engines are usually trained and evaluated against human scores and compared to the benchmark of human-human agreement. In this paper we compare the performance of an automated speech scoring engine using two corpora: a corpus of almost 700,000 randomly sampled spoken responses with scores assigned by one or two raters during operational scoring, and a corpus of 16,500 exemplar responses with scores reviewed by multiple expert raters. We show that the choice of corpus used for model evaluation has a major effect on estimates of system performance with r varying between 0.64 and 0.80. Surprisingly, this is not the case for the choice of corpus for model training: when the training corpus is sufficiently large, the systems trained on different corpora showed almost identical performance when evaluated on the same corpus. We show that this effect is consistent across several learning algorithms. We conclude that evaluating the model on a corpus of exemplar responses if one is available provides additional evidence about system validity; at the same time, investing effort into creating a corpus of exemplar responses for model training is unlikely to lead to a substantial gain in model performance.

This study aims to model the discourse structure of spontaneous spoken responses within the context of an assessment of English speaking proficiency for non-native speakers. Rhetorical Structure Theory (RST) has been commonly used in the analysis of discourse organization of written texts; however, limited research has been conducted to date on RST annotation and parsing of spoken language, in particular, non-native spontaneous speech. Due to the fact that the measurement of discourse coherence is typically a key metric in human scoring rubrics for assessments of spoken language, we conducted research to obtain RST annotations on non-native spoken responses from a standardized assessment of academic English proficiency. Subsequently, automatic parsers were trained on these annotations to process non-native spontaneous speech. Finally, a set of features were extracted from automatically generated RST trees to evaluate the discourse structure of nonnative spontaneous speech, which were then employed to further improve the validity of an automated speech scoring system.

Although yes/no questions are one of the most frequently occurring question types in English, research on the development and production of yes/no questions-in particular in young English learners-is still very limited. For example, we know very little about potential errors young L2 learners make when they produce yes/no questions-an area that is crucial in order to provide useful feedback in different learning environments, including computer-based applications. This paper reports on an exploratory study conducted with Can you guess who I am?, an interactive, spoken-dialogue-based speaking activity that allows young English learners to practice yes/no questions. After introducing the SDS-based speaking activity, we present the findings from a systematic investigation of the output produced by 27 young English learners in Germany (ages 9-11) who engaged with the activity. A particular focus in the analysis was placed on the types of yes/no questions elicited and the types of errors made by the young learners. The findings provide further empirical support for a six-stage framework for the development of question formation in L2 learners [14]. Moreover, they offer insights into the types of errors young EFL learners make in forming polar interrogatives such as systematic confusion with regard to the auxiliaries "to be" and "to do". The findings are discussed in terms of (a) how they contribute to a more comprehensive understanding of young learner's speech and (b) how they will be used to inform further development of more targeted feedback options that can be implemented into the SDSbased speaking activity in order to harness its full potential for L2 learning.

Hydration status and choline nutrition were evaluated relative to the concentration of lung surfactant phosphatidylcholine (PC) in fasted rats. Rats deprived of food for 72 h showed lower voluntary water consumption and consistently lower levels of both PC and total phospholipid (TPL) present in isolated pulmonary surfactant than ad libitum-fed controls, although the ratio of surfactant PC to TPL and the residual PC and TPL concentrations were not different. Higher hematocrit values observed in the fasted animals were not altered by the administration of water or saline by orogastric tube nor was the low surfactant PC level corrected by fluid therapy. Evidence of choline deficiency was demonstrated in the fasted rats as there was a significant shift in hepatic PC:phosphatidylethanolamine; however, plasma choline levels did not change. The administration to fasted animals of up to 2.4 mmol of choline chloride via the drinking water (containing 60 mM choline) or an orogastric tube did not affect the plasma choline concentrations or the production of lung surfactant PC.

This study aims to build an automatic system for the detection of plagiarized spoken responses in the context of an assessment of English speaking proficiency for non-native speakers. Classification models were trained to distinguish between plagiarized and nonplagiarized responses with two different types of features: text-to-text content similarity measures, which are commonly used in the task of plagiarism detection for written documents, and speaking proficiency measures, which were specifically designed for spontaneous speech and extracted using an automated speech scoring system. The experiments were first conducted on a large data set drawn from an operational English proficiency assessment across multiple years, and the best classifier on this heavily imbalanced data set resulted in an F1-score of 0.761 on the plagiarized class. This system was then validated on operational responses collected from a single administration of the assessment and achieved a recall of 0.897. The results indicate that the proposed system can potentially be used to improve the validity of both human and automated assessment of non-native spoken English.