Extracting audio summaries to support effective spoken document search

Abstract

We address the challenge of extracting query biased audio summaries from podcasts to support users in making relevance decisions in spoken document search via an audio-only communication channel. We performed a crowdsourced experiment that demonstrates that transcripts of spoken documents created using Automated Speech Recognition (ASR), even with significant errors, are effective sources of document summaries or "snippets" for supporting users in making relevance judgments against a query. In particular, the results show that summaries generated from ASR transcripts are comparable, in utility and user-judged preference, to spoken summaries generated from error-free manual transcripts of the same collection. We also observed that content-based audio summaries are at least as preferred as synthesized summaries obtained from manually curated metadata, such as title and description. We describe a methodology for constructing a new test collection, which we have made publicly available.

Accessing Media Via an Audio-only Communication Channel: A Log Analysis

“…Therefore it may be helpful for the users to hear their query words in the context of the found document. For podcasts, this may mean that users listen to a snippet extracted from the podcast audio in order to understand the context of their query word [12].…”

100,000 Podcasts: A Spoken English Document Corpus

“…Spoken document summaries are also available for the AMI meeting corpus (Mccowan et al, 2005) and the ICSI meeting corpus (Janin et al, 2003), as well as corpora of lectures (Miller, 2019), and voicemail (Koumpis and Renals, 2005). Spina et al (2017) collect and evaluate 217 hours of podcasts for query-biased extractive summarization. In recent work, Tardy et al (2020) train a model to reproduce full-length manual reports aligned with automatic speech recognition transcripts of meetings, and Gholipour Ghalandari et al (2020) generate a corpus for multi-document summarization.…”

CC-News-En

“…Similarly, the rise in popularity of spoken-text retrieval devices means that studying how searchers form queries after listening to an audio snippet will be useful. Spina et al [46] show that query-biased document summaries presented as audio are practical in conversational IR. Providing snippets through speech synthesis introduces more presentation factors, as Chuklin et al [18] note, where read-outs with prosody changes were subjectively more informative, at the expense of their aesthetic quality.…”