Interpolation of Lost Speech Segments Using LP-HNM Model With Codebook Post-Processing

Abstract

This paper presents a method for interpolation of lost speech segments. The interpolation method can be used for packet loss concealment in voice communication over mobile phones, for voice over IP or for restoration of lost segments in speech recordings. The interpolation method employs a combination of a linear prediction (LP) model of the spectral envelope and a harmonic noise model (HNM) of the excitation of speech. The speech interpolation problem is transformed to the modeling and interpolation of the trajectories of LP parameters and the amplitude, phase and harmonicity of HNM tracks of speech excitation. In particular, the interpolation of harmonicity results in a smooth transition from voiced to unvoiced speech and vice versa. Crucially, the proposed interpolation method does not suffer from the consequences of zero-excitation of conventional autoregressive (AR) interpolation. Different combinations of linear and autoregressive interpolation methods are evaluated for the estimation of the time-varying parameters of LP-HNM tracks. Furthermore, a post-processing codebook mapping, employed to enhance the interpolation of the spectral envelope of speech, results in improved output quality for longer length speech gaps. For different packet loss rates and patterns of distributions of missing speech gaps, the proposed interpolation methods are evaluated and compared with popular AR-based interpolation methods and the speech packet recovery method specified in the ITU G.711 standard, as a reference. The evaluation results show that the proposed methods substantially improve the restoration of formants and harmonic tracks and consistently results in significant performance gain and improved perceptual quality of speech.

Recovery of Lost Speech Segments Using Incremental Subspace Learning

“…The Gilbert model [3] is used to introduce frame loss in speech signals. After introducing the gaps in speech signals, each signal is reconstructed using the proposed algorithm.…”

“…In [2], Liang waveform substitution have proven to be very popular due to its simplicity. However, continual loss of segments leads to metallic-sounding artifacts [3].…”

Robust speech recognition using harmonic features

“…Voiced speech signals are built from harmonic components which are periodic and can be resynthesized easily. Many algorithms to extract harmonic components from speech signals have been proposed where the motivation for the intensive research is its wide usability within the field of engineering such as: speech analysis [1][2][3][4], speech coding [5][6][7], pitch tracking and estimation [3,7,8], speech enhancement using harmonic regeneration [9], post-speech enhancement [10,11], bandwidth extension [12,13] and voice activity detection [14]. Recently, the periodicity of harmonic structure has been used to compensate noise and to identify the corrupted time-frequency regions.…”

“…The resulting signals show an increased naturalness and perceptual quality in speech. Another new application of the harmonic model has been an add‐on module with several popular noise reduction methods [4]. The add‐on module utilises the harmonic plus noise model (HNM) of speech to retrieve damaged speech structure.…”