Specifying Challenges in Transcribing Covert Recordings: Implications for Forensic Transcription

There is a growing body of literature on forensic transcription of covert recordings obtained by clandestine law enforcement operations. Due to the nature of these operations, the quality of the recordings, particularly those obtained by planting listening devices in a car or a house, is often extremely poor. When tendering such recordings as evidence in court for prosecuting an alleged crime, a transcript will often accompany the recording to assist the triers of fact (i.e., judges and jurors) to hear better. In the context of multilingual and multicultural Australia, often such forensic recordings may contain languages other than English, and therefore a translation into English is required to facilitate understanding of the verbal exchanges in the recording. Little is known, however, about the people engaged by law enforcement to undertake these forensic translation tasks, what qualification and training they possess, how they carry out the tasks, and if there is a system to safeguard the quality and reliability of their translation output. This paper reports on the first ever survey conducted in Australia on professional interpreters and translators who have been engaged to perform this type of work. Descriptive statistics and thematic analysis of text answers provide a qualitative account of the status quo which has not been documented before. Deficiencies of the current practice and its associated risks are identified. Recommendations are proposed as the first step to address the issues identified.

Section: Concludingsupporting

confidence: 70%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Transcribing and translating forensic speech evidence containing foreign languages—An Australian perspective

Lai

2023

“…Many have assumed that this can be achieved by individual experts evaluating police transcripts -as I did myself until casework experience led me to argue this it is not suitable, for a range of reasons (Fraser, 2020b). These reasons have recently been amplified by a ground-breaking study (Love and Wright, 2021) in which eight different (expert) transcribers of indistinct audio created eight transcripts that differ in substantial ways. The point is that the experts were operating under uncertainty regarding the true content of the audio.…”

Section: Discussionmentioning

confidence: 99%

A Framework for Deciding How to Create and Evaluate Transcripts for Forensic and Other Purposes

Fraser

2022

Transcripts are used successfully in many areas of contemporary society. However, some uses of transcripts show systemic problems, with significant negative consequences. The key to finding effective solutions in these areas is to determine which factors contribute most strongly to the problems – which may be different from those to which they are commonly ascribed. This systematic review offers a conceptual framework for understanding the nature of transcripts in general, and the factors that contribute to a transcript's reliability and suitability for purpose. It then demonstrates how the framework can explain the (mostly) successful use of transcripts in two domains: court proceedings and linguistics research. Next, it uses the framework to examine two problematic cases: transcripts of forensic audio used as evidence in criminal trials, and transcripts of police interviews with suspects. A crucial observation is that, while it is common, and understandable, to focus on the transcriber as the source of problems with transcripts, transcription is actually a complex process involving practitioners in multiple roles, of which the transcriber role is not always the most important. Solving problems thus requires coordination of a range of factors. The analysis ends with practical suggestions for how to seek solutions for both the problematic areas reviewed, with attention to the role that linguistic science needs to play. The conclusion amplifies recent calls to consolidate transcription as a dedicated field of study within linguistics.

“…Since Loakes (2022) addressed the issue of how ASR copes with indistinct forensic-like recordings, some new work in this space has been conducted with the newer generation of ASR systems which further demonstrates some of the issues discussed above; however, this new research has not made use of Whisper. Similar to Harrington et al (2022), Loakes (2022) carried out a comparison of various ASR systems with recordings known to be difficult for human transcribers, as reported by Love and Wright (2021). They used 18 British English utterances of which they could be certain of the content and used 12 commercially available ASR systems to compare how well the systems transcribed forensic-like audio.…”

Section: Introductionmentioning

confidence: 99%

Automatic speech recognition and the transcription of indistinct forensic audio: how do the new generation of systems fare?

Loakes

2024

This study provides an update on an earlier study in the “Capturing Talk” research topic, which aimed to demonstrate how automatic speech recognition (ASR) systems work with indistinct forensic-like audio, in comparison with good-quality audio. Since that time, there has been rapid technological advancement, with newer systems having access to extremely large language models and having their performance proclaimed as being human-like in accuracy. This study compares various ASR systems, including OpenAI’s Whisper, to continue to test how well automatic speaker recognition works with forensic-like audio. The results show that the transcription of a good-quality audio file is at ceiling for some systems, with no errors. For the poor-quality (forensic-like) audio, Whisper was the best performing system but had only 50% of the entire speech material correct. The results for the poor-quality audio were also generally variable across the systems, with differences depending on whether a .wav or .mp3 file was used and differences between earlier and later versions of the same system. Additionally, and against expectations, Whisper showed a drop in performance over a 2-month period. While more material was transcribed in the later attempt, more was also incorrect. This study concludes that forensic-like audio is not suitable for automatic analysis.