Aligning Very Long Speech Signals to Bilingual Transcriptions of Parliamentary Sessions

Bordel, Germán; Peñagarikano, Mikel; Rodríguez-Fuentes, Luis Javier; Fernández, María Amparo Varona

doi:10.1007/978-3-642-35292-8_8

Cited by 2 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The subtitles are generated by applying a heuristic approach to cut the aligned minutes into reasonably short segments so that the subtitles can be easily read. This is how we have been generating the subtitles of the Basque Parliament since 2010 to the present [35][36][37][38].…”

Section: Processing Text Resourcesmentioning

confidence: 99%

“…Since audio chunks will be tokenized in terms of phonetic units, alignments must be performed at the phonetic level. Thus, the text to be aligned with each audio chunk is passed through an in-house bilingual grapheme-to-phoneme (G2P) converter [36,37] to obtain a sequence of phonetic units (the reference sequence). The G2P converter is based on two dynamic dictionaries for Basque and Spanish, initialized on Wikipedia and updated with each new word found in transcriptions.…”

Section: Text Tokenizationmentioning

confidence: 99%

“…This alignment is based on the principle of maximizing the number of matches, which generally equates to minimizing the occurrences of deletions, insertions and substitutions. This approach follows the same text-and-speech alignment methodology that our group has successfully implemented since 2010 for synchronizing subtitles with spoken content in the Basque Parliament [35][36][37][38]. As a result, areas with a high concentration of alignment errors likely indicate discrepancies between the draft minutes and the actual audio, suggesting that these segments should be omitted from the training dataset.…”

Section: Alignmentmentioning

confidence: 99%

See 2 more Smart Citations

A Bilingual Basque–Spanish Dataset of Parliamentary Sessions for the Development and Evaluation of Speech Technology

Varona,

Penagarikano,

Bordel

et al. 2024

Applied Sciences

Self Cite

View full text Add to dashboard Cite

The development of speech technology requires large amounts of data to estimate the underlying models. Even when relying on large multilingual pre-trained models, some amount of task-specific data on the target language is needed to fine-tune those models and obtain competitive performance. In this paper, we present a bilingual Basque–Spanish dataset extracted from parliamentary sessions. The dataset is designed to develop and evaluate automatic speech recognition (ASR) systems but can be easily repurposed for other speech-processing tasks (such as speaker or language recognition). The paper first compares the two target languages, emphasizing their similarities at the acoustic-phonetic level, which sets the basis for sharing data and compensating for the relatively small amount of spoken resources available for Basque. Then, Basque Parliament plenary sessions are characterized in terms of organization, topics, speaker turns and the use of the two languages. The paper continues with the description of the data collection procedure (involving both speech and text), the audio formats and conversions along with the creation and postprocessing of text transcriptions and session minutes. Then, it describes the semi-supervised iterative procedure used to cut, rank and select the training segments and the manual supervision process employed to produce the test set. Finally, ASR experiments are presented using state-of-the-art technology to validate the dataset and to set a reference for future works. The datasets, along with models and recipes to reproduce the experiments reported in the paper, are released through Hugging Face.

show abstract

Section: Processing Text Resourcesmentioning

confidence: 99%

Section: Text Tokenizationmentioning

confidence: 99%

Section: Alignmentmentioning

confidence: 99%

See 1 more Smart Citation

A Bilingual Basque–Spanish Dataset of Parliamentary Sessions for the Development and Evaluation of Speech Technology

Varona,

Penagarikano,

Bordel

et al. 2024

Applied Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

“…An in-house bilingual grapheme-to-phoneme (G2P) converter has been developed and applied to obtain the phonetic baseforms of words in Basque and Spanish [27,28]. All words are then gathered in a single lexicon.…”

Section: Lexical Modelsmentioning

confidence: 99%

“…On the other hand, the minutes are passed through the above mentioned G2P converter to acquire a reference (nominal) sequence of phonetic units. Finally, the recognized and reference sequences of phonetic units are aligned one with another under the criterion of maximizing the number of matching units (which is approximately the same as minimizing the number of deletions, insertions and substitutions), following the same text-and-speech alignment method that has been successfully applied in our group for the alignment of BP subtitles [28,34,35]. In this way, those regions showing a high density of errors in the alignment would correspond to parts of the minutes which do not match the audio contents.…”

Section: Iterative Data Collection Through Phonetic Decoding and Alig...mentioning

confidence: 99%

Semisupervised Speech Data Extraction from Basque Parliament Sessions and Validation on Fully Bilingual Basque–Spanish ASR

et al. 2023

Self Cite

View full text Add to dashboard Cite

In this paper, a semisupervised speech data extraction method is presented and applied to create a new dataset designed for the development of fully bilingual Automatic Speech Recognition (ASR) systems for Basque and Spanish. The dataset is drawn from an extensive collection of Basque Parliament plenary sessions containing frequent code switchings. Since session minutes are not exact, only the most reliable speech segments are kept for training. To that end, we use phonetic similarity scores between nominal and recognized phone sequences. The process starts with baseline acoustic models trained on generic out-of-domain data, then iteratively updates the models with the extracted data and applies the updated models to refine the training dataset until the observed improvement between two iterations becomes small enough. A development dataset, involving five plenary sessions not used for training, has been manually audited for tuning and evaluation purposes. Cross-validation experiments (with 20 random partitions) have been carried out on the development dataset, using the baseline and the iteratively updated models. On average, Word Error Rate (WER) reduces from 16.57% (baseline) to 4.41% (first iteration) and further to 4.02% (second iteration), which corresponds to relative WER reductions of 73.4% and 8.8%, respectively. When considering only Basque segments, WER reduces on average from 16.57% (baseline) to 5.51% (first iteration) and further to 5.13% (second iteration), which corresponds to relative WER reductions of 66.7% and 6.9%, respectively. As a result of this work, a new bilingual Basque–Spanish resource has been produced based on Basque Parliament sessions, including 998 h of training data (audio segments + transcriptions), a development set (17 h long) designed for tuning and evaluation under a cross-validation scheme and a fully bilingual trigram language model.

show abstract

Aligning Very Long Speech Signals to Bilingual Transcriptions of Parliamentary Sessions

Cited by 2 publications

References 8 publications

A Bilingual Basque–Spanish Dataset of Parliamentary Sessions for the Development and Evaluation of Speech Technology

A Bilingual Basque–Spanish Dataset of Parliamentary Sessions for the Development and Evaluation of Speech Technology

Semisupervised Speech Data Extraction from Basque Parliament Sessions and Validation on Fully Bilingual Basque–Spanish ASR

Contact Info

Product

Resources

About