A Framework for Deciding How to Create and Evaluate Transcripts for Forensic and Other Purposes

Abstract

Transcripts are used successfully in many areas of contemporary society. However, some uses of transcripts show systemic problems, with significant negative consequences. The key to finding effective solutions in these areas is to determine which factors contribute most strongly to the problems – which may be different from those to which they are commonly ascribed. This systematic review offers a conceptual framework for understanding the nature of transcripts in general, and the factors that contribute to a transcript's reliability and suitability for purpose. It then demonstrates how the framework can explain the (mostly) successful use of transcripts in two domains: court proceedings and linguistics research. Next, it uses the framework to examine two problematic cases: transcripts of forensic audio used as evidence in criminal trials, and transcripts of police interviews with suspects. A crucial observation is that, while it is common, and understandable, to focus on the transcriber as the source of problems with transcripts, transcription is actually a complex process involving practitioners in multiple roles, of which the transcriber role is not always the most important. Solving problems thus requires coordination of a range of factors. The analysis ends with practical suggestions for how to seek solutions for both the problematic areas reviewed, with attention to the role that linguistic science needs to play. The conclusion amplifies recent calls to consolidate transcription as a dedicated field of study within linguistics.

Automatic speech recognition and the transcription of indistinct forensic audio: how do the new generation of systems fare?

“…The potential for such a large error rate (50% WER at best) is not appropriate for forensic contexts; a transcription in which only 50% is correct is not useable. While the results of this study, for Whisper in particular, are a marked improvement in performance compared to the systems trialled on the same audio in Loakes (2022), this study advocates for the use of human transcription done in a measured and systematic manner (e.g., Fraser, 2022, also Loakes, 2022Harrington, 2023) and for keeping ASR methods limited to tasks they are designed for. This aligns with the findings from Harrington (2023) discussed earlier, who observed that it is more efficient to do a transcription from scratch than to try and use the output of ASR systems which contain relatively high error rates.…”

“…Forensic audio is audio that is generally captured in high stakes and often criminal contexts. This type of audio is defined by Fraser (2022): 8, as …speech that has been captured, typically in a covert (secret) recording obtained as part of a criminal investigation, and is later used as evidence in a trial. Such recordings provide powerful evidence, allowing the court to hear speakers making admissions they would not make openly.…”

Transcribing and translating forensic speech evidence containing foreign languages—An Australian perspective

“…The two-step process recommended by best practice FTT (see discussion in 3.1) points to two areas of specialist training required: transcription (from spoken LOTE to written LOTE), and translation (from LOTE into English). The need for training for the former is no different from monolingual settings, which has been advocated by scholars (Fraser and Stevenson, 2014;Romito, 2017;Fraser and Loakes, 2020;Fraser, 2021Fraser, , 2022 in order to achieve accuracy and reliability in forensic contexts. The current study reveals the fact that the majority of the respondents undertook very infrequent FTT assignments, and unlike other areas such as community interpreting for healthcare, education, or social services, FTT does not constitute their bread and butter.…”

Forensic Linguistics: A Concise Depiction of Key Elements

“…Forensic linguistics is now used in a wide range of legal contexts that starts from speech and audio analysis, authorship distribution and others that has made it more reliable in contemporary times [19]. The other theme of this study was to determine significance of appropriate key elements in linguistics for appropriate evaluation for forensics.…”