Speech recognition evaluation: a review of the U.S. CSR and LVCSR programmes

Young, Steve; Chase, Lin Lawrence

doi:10.1006/csla.1998.0101

Cited by 21 publications

(8 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Speech recognition, and especially continuous speech recognition, has been a major (D)ARPA concern, with high-profile evaluations since the eighties (Young and Chase 1998). As in the TREC case, the evaluations have done much to improve and consolidate SR technology; indeed the claims for the evaluations as agents (along with more machine power) in raising performance levels, and in accelerating technology development, have been much stronger than in the TREC case.…”

Section: Speech Recognition (Sr)mentioning

confidence: 99%

Automatic language and information processing: rethinking evaluation

Jones

2001

Nat. Lang. Eng.

View full text Add to dashboard Cite

System evaluation has mattered since research on automatic language and information processing began. But the (D)ARPA conferences have raised the stakes substantially in requiring and delivering systematic evaluations and in sustaining these through long term programmes; and it has been claimed that this has both significantly raised task performance, as defined by appropriate effectiveness measures, and promoted relevant engineering development. These controlled laboratory evaluations have made very strong assumptions about the task context. The paper examines these assumptions for six task areas, considers their impact on evaluation and performance results, and argues that for current tasks of interest, e.g. summarising, it is now essential to play down the present narrowly-defined performance measures in order to address the task context, and specifically the role of the human participant in the task, so that new measures, of larger value, can be developed and applied. * I am grateful to James Allan and Fred Jelinek for inviting me to give the talk on which this paper is based, and to two referees for their comments.

show abstract

Section: Speech Recognition (Sr)mentioning

confidence: 99%

Automatic language and information processing: rethinking evaluation

Jones

2001

Nat. Lang. Eng.

View full text Add to dashboard Cite

show abstract

“…Starting with the artificial 1000 word Resource Management task [140], the technology developed rapidly and by the mid-1990's, reasonable accuracy was being achieved for unrestricted speaker independent dictation. Much of this development was driven by a series of DARPA and NSA programmes [188] which set ever more challenging tasks culminating most recently in systems for multilingual transcription of broadcast news programmes [134] and for spontaneous telephone conversations [62].…”

Section: Introductionmentioning

confidence: 99%

The Application of Hidden Markov Models in Speech Recognition

Gales

Young

2007

FNT in Signal Processing

427

155

View full text Add to dashboard Cite

Hidden Markov Models (HMMs) provide a simple and effective framework for modelling time-varying spectral vector sequences. As a consequence, almost all present day large vocabulary continuous speech recognition (LVCSR) systems are based on HMMs.Whereas the basic principles underlying HMM-based LVCSR are rather straightforward, the approximations and simplifying assumptions involved in a direct implementation of these principles would result in a system which has poor accuracy and unacceptable sensitivity to changes in operating environment. Thus, the practical application of HMMs in modern systems involves considerable sophistication.The aim of this review is first to present the core architecture of a HMM-based LVCSR system and then describe the various refinements which are needed to achieve state-of-the-art performance. These refinements include feature projection, improved covariance modelling, discriminative parameter estimation, adaptation and normalisation, noise compensation and multi-pass system combination. The review concludes with a case study of LVCSR for Broadcast News and Conversation transcription in order to illustrate the techniques described.

show abstract

“…This procedure has an especially long tradition in the field of automatic speech recognition (cf. [82,254]), where several standardized test scenarios were defined. Nevertheless, also in this area it is extremely difficult to compare systems by the evaluation results documented.…”

Section: Introductory Remarksmentioning

confidence: 99%

Markov Models for Pattern Recognition

Fink¹

2008

View full text Add to dashboard Cite

Speech recognition evaluation: a review of the U.S. CSR and LVCSR programmes

Cited by 21 publications

References 10 publications

Automatic language and information processing: rethinking evaluation

Automatic language and information processing: rethinking evaluation

The Application of Hidden Markov Models in Speech Recognition

Markov Models for Pattern Recognition

Contact Info

Product

Resources

About