To show or not to show: The effects of item stems and answer options on performance on a multiple-choice listening comprehension test

Yanagawa, Kozo; Green, Anthony

doi:10.1016/j.system.2007.12.003

Cited by 57 publications

(47 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…) in a passage on item difficulty was examined in Yanagawa and Green (2008). Their results showed that the presence of infrequent vocabulary in a text affected item difficulty, but unlike the findings from Nissan et al (1996), it was found that less-frequent vocabulary in the passage was associated with easier items.…”

Section: Infrequent Wordsmentioning

confidence: 96%

“…While the results from Nissan et al (1996) and Kostin (2004) imply that the presence of two or more negatives may increase item difficulty, and those of Yanagawa and Green (2008) suggest that negatives may increase the difficulty of particular kinds of items, Ying-hui (2006) failed to find any effect of negatives on difficulty. None of the studies investigating referentials or dependent clauses found a relationship between these factors and item difficulty, so there is no evidence to date that these factors will impact L2 listening comprehension.…”

Section: Summary: Syntactic Complexitymentioning

confidence: 97%

“…The studies that have investigated the impact of the syntactic structure of a passage have done so by considering the degree of subordination (Blau, 1990;Cervantes & Gainer, 1992;Pica et al, 1987), the number of negatives (Kostin, 2004;Nissan et al, 1996;Yanagawa & Green, 2008;Ying-hui, 2006), the number of dependent clauses (Kostin, 2004;Ying-hui, 2006), or the number of references (Kostin, 2004;Ying-hui, 2006) in the passage.…”

Section: Syntactic Complexitymentioning

confidence: 99%

“…Thus, it is possible that earlier studies examining the relationship between vocabulary frequency and item difficulty were inadvertently measuring the effect of infrequent information that was necessary for answering the item rather than overall frequency of words in the passage. Yanagawa and Green (2008) uncovered conflicting results, but the authors themselves argued that this finding might have been a result of the word list they used to determine frequency. However, all the studies discussed in this section have the limitation of using word lists to determine word frequency that may not be as reliable as originally expected; this limitation should be taken into consideration when interpreting the findings.…”

Section: Infrequent Wordsmentioning

confidence: 99%

“…Other, similar analyses of test passages failed to uncover any overall relationship between negations in the passage and item difficulty (Yanagawa & Green, 2008;Ying-hui, 2006). However, Yanagawa and Green (2008) did find that negatives affected item difficulty for certain items, such that items where the correct answer contained many of the same words as the passage were more difficult when there were more negatives in the passage, and items where an incorrect answer contained a lot of the same words as the passage were easier when there were more negatives.…”

mentioning

confidence: 96%

See 4 more Smart Citations

What makes listening difficult? Factors affecting second language listening comprehension

Bloomfield¹,

Wayland²,

Rhoades³

et al. 2010

121

120

View full text Add to dashboard Cite

Factors affecting second language listening comprehension gies-that is, those who are aware of and use effective strategies, such as avoiding mental translation-demonstrate better L2 listening comprehension. 2 In addition to these general cognitive abilities, a number of factors pertaining to experience with the L2 influence listening skill. These factors include the amount of prior exposure to the language; familiarity with and an ability to understand the non-native language's phonology; vocabulary size; and background knowledge about the topic, text, structure, schema, and culture.Familiarity with the L2 changes the extent to which the L2 listener uses top-down or bottom-up strategies in listening. For example, expert listeners use both types of strategies: They are able to accurately make sense of the speech signal (bottom-up information) 3 and integrate this information with PurPose-To establish what is currently known about factors that affect foreign language listening comprehension, with a focus on characteristics of the listener, passage, and testing conditions.ConClusions-Research on second language (L2) listening comprehension strongly supports the importance of a number of factors, for example, a listener's working memory capacity and the number of ideas in a passage. Much of the research, however, reports weak or inconclusive results, leaving many factors and complex interactions among factors unresolved and in need of further investigation.relevanCe-Identifying the factors that affect L2 listening comprehension will help Defense Language Institute Proficiency Test (DLPT) designers anticipate how qualities of selected authentic materials will impact listening comprehension. Executive Summary PurPoseThe U.S. Government administers the Defense Language Proficiency Test (DLPT) to military linguists and other government personnel to assess their listening and reading comprehension in a number of foreign languages, including critical languages such as Mandarin, Modern Standard Arabic, Egyptian Arabic, and Persian Farsi. The DLPT is updated every 10 to 15 years, and the most recent transition-from DLPT IV to DLPT5-included a greater emphasis on testing listening comprehension with authentic materials. In turn, this has led to a growing interest in the factors that make second language (L2) listening difficult.To examine these factors, CASL reviewed the current scientific literature and summarized the characteristics of listeners, passages, and testing conditions. The review targeted features of particular interest to stakeholders at the Defense Language Institute (DLI). The long-term goal of the project is to support the selection of authentic listening materials that accurately reflect different proficiency levels. ConClusionsAlthough the available research on L2 listening comprehension is limited, CASL's literature review identified several factors that affect listening comprehension. These factors are summarized below and in Tables 1, 2, and 3. 1 Characteristics of the listenerUnderstanding a fo...

show abstract

Section: Infrequent Wordsmentioning

confidence: 96%

Section: Summary: Syntactic Complexitymentioning

confidence: 97%

Section: Syntactic Complexitymentioning

confidence: 99%

Section: Infrequent Wordsmentioning

confidence: 99%

mentioning

confidence: 96%

See 3 more Smart Citations

What makes listening difficult? Factors affecting second language listening comprehension

Bloomfield¹,

Wayland²,

Rhoades³

et al. 2010

121

120

View full text Add to dashboard Cite

show abstract

Assessing Listening

Wagner

2013

The Companion to Language Assessment

View full text Add to dashboard Cite

This chapter gives an overview of important issues related to second language (L2) listening assessment. The first part of the chapter focuses on the unique challenges the listening mode presents to teachers and test developers, framed within the context of two separate but related principles of language testing: construct validity, and target language use (TLU) domain and task characteristics. The chapter then reviews the research on a number of practical issues in L2 listening assessment: using integrated test tasks; including linguistics features characteristic of unplanned spoken discourse in the spoken input; selecting the different types and varieties of the spoken language input; using the visual channel; choosing different item types and response formats; and presenting the listening texts and test questions to test takers. The chapter then examines issues of test consequences and washback in L2 listening assessment, and concludes with an examination of future directions in the area.

show abstract

Assessment of Listening

Ockey¹

2012

The Encyclopedia of Applied Linguistics

View full text Add to dashboard Cite

Listening is important for in‐person communication with estimates that it accounts for more than 45% of the time spent communicating (Feyten, 1991). Listening continues to become more important in virtual environments with the increase of communication through such technologies as video broadcasting and Skype. It follows that teaching and assessing the listening skill of second language learners is essential. Unfortunately, the assessment of second language listening comprehension has attracted little research attention (Buck, 2001), and as a result, understanding of how to best assess it is limited.

show abstract

To show or not to show: The effects of item stems and answer options on performance on a multiple-choice listening comprehension test

Cited by 57 publications

References 8 publications

What makes listening difficult? Factors affecting second language listening comprehension

What makes listening difficult? Factors affecting second language listening comprehension

Assessing Listening

Assessment of Listening

Contact Info

Product

Resources

About