No abstract

The current study investigated linguistic influences on comprehensibility (ease of understanding) and accentedness (linguistic nativelikeness) in second language (L2) learners' extemporaneous speech. Target materials included picture narratives from 40 native French speakers of English from different proficiency levels. The narratives were subsequently rated by 20 native speakers with or without linguistic and pedagogical experience for comprehensibility, accentedness, and 11 linguistic variables spanning the domains of phonology, lexis, grammar, and discourse structure. Results showed that comprehensibility was associated with several linguistic variables (vowel/consonant errors, word stress, fluency, lexis, grammar), whereas accentedness was chiefly linked to pronunciation (vowel/consonant errors, word stress). Native speaking listeners thus appear to pay particular attention to pronunciation, rather than lexis and grammar, to evaluate nativelikeness but tend to consider various sources of linguistic information in L2 speech in judging comprehensibility. The use of listener ratings (perceptual measures) in evaluating linguistic aspects of learner speech and their implications for language assessment and pedagogy are discussed.

This mixed-methods study examines the effects of rating scale length and rater experience on listeners' judgments of second language (L2) speech. Twenty experienced and 20 novice raters, who were randomly assigned to 5-point or 9-point rating scale conditions, judged speech samples of 38 newcomers to Canada on numerical rating scales for comprehensibility, accentedness, and fluency. Results yielded high Cronbach's alpha coefficients and no group differences for rating scale length or rater experience. However, Rasch category probability plots revealed that raters had difficulty differentiating between scale steps, particularly in mid-scale range-a challenge which was exacerbated in the 9-point scale condition. Evidence from verbal protocols and post-task interviews suggested that experienced and novice raters adopted strategies to either draw on or offset their perceived experience with L2 speech in conducting their ratings. Implications for L2 pronunciation research are discussed, including the need to draw on raters' perceptions to develop a greater understanding of major listener perceptual constructs in L2 pronunciation and to move beyond adherence to the native speaking standard in data collection procedures. L2 pronunciation conventions 3 Rater experience, rating scale length, and judgments of L2 pronunciation: Revisiting research conventions Rating scales provide the framework within which human raters score second language (L2) performance, which is taken to be an indicator of L2 learners' ability on the construct being measured. Such assessment schemes constrain, structure, or filter raters' responses, often through rigid scale descriptors associated with a fixed number of scale bands. Lumley describes the tension between the "simplified orderliness of the rating scale," which underrepresents the complexity involved in L2 performance, and raters' unconstrained reactions to the performance, which may be disordered and complex (2005, p. 248). The challenge for raters is to reconcile their possibly idiosyncratic, intuitive, or non-linear impressions of an L2 performance with rating scale specifications, including discretizing what might be a "grey area" performance. In L2 pronunciation research, 9-point numerical rating scales are increasingly becoming entrenched as a research convention to measure qualitative constructs of comprehensibility, accentedness, and fluency. 1 Nine levels are used due to the conclusion that, for accentedness at least, smaller scales might cause a ceiling effect (Southwood & Flege, 1999). Generally, only scalar endpoints are defined (e.g., very easy/difficult to understand for comprehensibility), with little guidance for raters on how to assess these dimensions other than to use the full range of the scale (Derwing, Munro, & Wiebe, 1998). A major appeal of these Likert-type scales to L2 pronunciation researchers is that their generic nature offers versatility, in that they can be used with learners from virtually any first language (L1) background or proficiency level on any task....

The current project aimed to investigate the potentially different linguistic correlates of comprehensibility (i.e., ease of understanding) and accentedness (i.e., linguistic nativelikeness) in adult second language (L2) learners' extemporaneous speech production. Timed picture descriptions from 120 beginner, intermediate, and advanced Japanese learners of English were analyzed using native speaker global judgments based on learners' comprehensibility and accentedness, and then submitted to segmental, prosodic, temporal, lexical, and grammatical analyses. Results showed that comprehensibility was related to all linguistic domains, and accentedness was strongly tied with pronunciation (specifically segmentals) rather than lexical and grammatical domains. In particular, linguistic correlates of L2 comprehensibility and accentedness were found to vary by learners' proficiency levels. In terms of comprehensibility, optimal rate of speech, appropriate and rich vocabulary use, and adequate and varied prosody were important for beginner to intermediate levels, whereas segmental accuracy, good prosody, and correct grammar featured strongly for intermediate to advanced levels. For accentedness, grammatical complexity was a feature of intermediate to high-level performance, whereas segmental and prosodic variables were essential to accentedness across all levels. These findings suggest that syllabi tailored to learners' proficiency level (beginner, intermediate, or advanced) and learning goal (comprehensibility or nativelike accent) would be advantageous for the teaching of L2 speaking.As many second language (L2) researchers have pointed out, it is crucial to set realistic goals for adult L2 learners, prioritizing understanding over © Cambridge University Press 2015 0142-7164/15

ABSTRACT The current study investigated task effects on listener perception of second language (L2) comprehensibility (ease of understanding). Sixty university-level adult speakers of English from 4 first language (L1) backgrounds (Chinese, Romance, Hindi, Farsi), with 15 speakers per group, were recorded performing 2 tasks (IELTS long-turn speaking task, TOEFL iBT integrated listening/reading and speaking task). The speakers' audio recordings were evaluated using continuous sliding scales by 10 native English listeners for comprehensibility as well as for 10 linguistic variables drawn from the domains of pronunciation, fluency, lexis, grammar, and discourse. In the IELTS task, comprehensibility was associated solely with pronunciation and fluency categories (specifically, segmentals, word stress, rhythm, and speech rate), with the Farsi group being the only exception. However, in the cognitively more demanding TOEFL iBT integrated task, in addition to pronunciation and fluency variables, comprehensibility was also linked to several categories at the level of grammar, lexicon, and discourse for all groups. In both 1 tasks, the relative strength of obtained associations also varied as a function of the speakers' L1. Results overall suggest that both task and speakers' L1 play important roles in determining ease of understanding for the listener, with implications for pronunciation teaching in mixed L1 classrooms and for operationalizing the construct of comprehensibility in assessments. END ABSTRACT

Link to this article: http://journals.cambridge.org/abstract_S1366728914000832How to cite this article: PAVEL TROFIMOVICH, TALIA ISAACS, SARA KENNEDY, KAZUYA SAITO and DUSTIN CROWTHER (2016 People are famously poor at judging their own ability, engaging in such behaviours as "errors of omission", "flawed self-assessment", and "faulty self-awareness"

Comprehensibility, a major concept in second language (L2) pronunciation research that denotes listeners’ perceptions of how easily they understand L2 speech, is central to interlocutors’ communicative success in real-world contexts. Although comprehensibility has been modeled in several L2 oral proficiency scales—for example, the Test of English as a Foreign Language (TOEFL) or the International English Language Testing System (IELTS)—shortcomings of existing scales (e.g., vague descriptors) reflect limited empirical evidence as to which linguistic aspects influence listeners’ judgments of L2 comprehensibility at different ability levels. To address this gap, a mixed-methods approach was used in the present study to gain a deeper understanding of the linguistic aspects underlying listeners’ L2 comprehensibility ratings. First, speech samples of 40 native French learners of English were analyzed using 19 quantitative speech measures, including segmental, suprasegmental, fluency, lexical, grammatical, and discourse-level variables. These measures were then correlated with 60 native English listeners’ scalar judgments of the speakers’ comprehensibility. Next, three English as a second language (ESL) teachers provided introspective reports on the linguistic aspects of speech that they attended to when judging L2 comprehensibility. Following data triangulation, five speech measures were identified that clearly distinguished between L2 learners at different comprehensibility levels. Lexical richness and fluency measures differentiated between low-level learners; grammatical and discourse-level measures differentiated between high-level learners; and word stress errors discriminated between learners of all levels.

This study critically examined the previously reported partial independence between second language (L2) accentedness (degree to which L2 speech differs from the target variety) and comprehensibility (ease of understanding). In prior work, comprehensibility was linked to multiple linguistic dimensions of L2 speech (phonology, fluency, lexis, grammar) whereas accentedness was narrowly associated with L2 phonology. However, these findings stemmed from a single task (picture narrative), suggesting that task type could affect the particular linguistic measures distinguishing comprehensibility from accentedness. To address this limitation, speech ratings of 10 native listeners assessing 60 speakers of L2 English in three tasks (picture narrative, IELTS, TOEFL) were analyzed, targeting two global ratings (accentedness, comprehensibility) and 10 linguistic measures (segmental and word stress accuracy, intonation, rhythm, speech rate, grammatical accuracy and complexity, lexical richness and complexity, discourse richness). Linguistic distinctions between accentedness and comprehensibility were less pronounced in the cognitively complex task (TOEFL), with overlapping sets of phonology, lexis, and grammar variables contributing to listener ratings of accentedness and comprehensibility. This finding points to multifaceted, task-specific relationships between these two constructs. ACCENTEDNESS AND COMPREHENSIBILITY ACROSS TASKS 1 AbstractThis study critically examined the previously reported partial independence between second language (L2) accentedness (degree to which L2 speech differs from the target variety) and