Deception detection in speech using bark band and perceptually significant energy features

Sanaullah, Muhammad; Gopalan, K.

doi:10.1109/mwscas.2013.6674872

Cited by 15 publications

(18 citation statements)

References 5 publications

(5 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They also ran tests with the CSC corpus using a combination of acoustic GMM and prosodic/lexical SVM with similar accuracy results. Using a (18) combined prosodic and non-linear dynamic (NLD) features as input to detect deception from speech signals [14]. Table I shows the different methods or algorithms used in previous research work to test unique speech features and how accurately they recognize deception.…”

Section: Previous Research Workmentioning

confidence: 99%

“…Each frame was divided into 21 Bark Bands within a 7700Hz frequency, as seen in Table 3. Spectral energy over the Bark scale is more natural in approximating the perception in the ear [18]. Critical bandwidth tends to remain constant (about 100 Hz) up to 500 Hz and increases to approximately 20 percent of the center frequency above 500 Hz [18].…”

Section: B Spectral Feature Development and Extractionmentioning

confidence: 99%

“…Spectral energy over the Bark scale is more natural in approximating the perception in the ear [18]. Critical bandwidth tends to remain constant (about 100 Hz) up to 500 Hz and increases to approximately 20 percent of the center frequency above 500 Hz [18]. The critical bandwidth [24] is calculated approximately using (4) where frequency f is in Hz.…”

Section: B Spectral Feature Development and Extractionmentioning

confidence: 99%

“…Frequency in Hz is converted to the Bark scale for analysis purposes. Each frame is normalized with the global masking threshold, and power spectral density to extract the spectral energy feature [18]. For reference purposes, the quiet threshold for hearing is obtained.…”

Section: B Spectral Feature Development and Extractionmentioning

confidence: 99%

“…In many speech processing tasks, deep neural networks have been successfully used in speaker verification [27], [28], speech enhancement [29], [30], [31], and speech recognition [32], [33], [34], deception detection [6], [18], and emotion recognition [13]. Spectral and cepstrum features and the target result of 'True' and 'False' speech were applied to a Levenberg-Marquardt neural network model and an LSTM neural network model for training.…”

Section: Feature Matchingmentioning

confidence: 99%

See 4 more Smart Citations

Use of Machine Learning for Deception Detection From Spectral and Cepstral Features of Speech Signals

Fernandes

Ullah

2021

IEEE Access

View full text Add to dashboard Cite

In this research, four unique nonlinear speech features are extracted and analyzed to study the dissimilarity pattern between when the speaker is being deceitful and truthful based on how human speech is perceived. The speaker was under stress in a police interrogation where two ground truth and two deceitful responses were recorded during three different times of the day. Using the audio recordings from all three sessions, the cepstral features and spectral energy features are extracted. Cepstral features are the Mel frequency cepstrum coefficient, from where the delta cepstrum and the time-difference cepstrum features are developed. On the other hand, the spectral energy features are the energy of Bark band energy from where the delta energy and the time-difference energy features are developed. The Levenberg-Marquardt classification method and the long short-term memory classification method are then applied to evaluate the accuracy of detecting deception based on the nine unique training and testing combinations of the three different sessions and their extracted cepstrum and spectral energy features. In addition, the principal component analysis is applied to reduce the dimensionality from the extracted features for further improvement. The projected principal components of the four types of features showed improved accuracy in order to distinguish between truthful and deceptive speech pattern. After incorporating with principal component analysis, the long short-term memory classification method with time-difference spectral energy feature shows the highest recognition rate compared to Levenberg-Marquardt algorithm with other cepstral and spectral features.

show abstract

Section: Previous Research Workmentioning

confidence: 99%

Section: B Spectral Feature Development and Extractionmentioning

confidence: 99%

Section: B Spectral Feature Development and Extractionmentioning

confidence: 99%

Section: B Spectral Feature Development and Extractionmentioning

confidence: 99%

Section: Feature Matchingmentioning

confidence: 99%

See 3 more Smart Citations

Use of Machine Learning for Deception Detection From Spectral and Cepstral Features of Speech Signals

Fernandes

Ullah

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

Investigation of Speech Signal Parameters Reflecting the Truth of Transmitted Information

Budkov

Vatamaniuk

Basov

et al. 2016

Speech and Computer

View full text Add to dashboard Cite

FacialCueNet: unmasking deception - an interpretable model for criminal interrogation using facial expressions

Nam,

Kim,

Bark

et al. 2023

Appl Intell

View full text Add to dashboard Cite

Polygraphs are used in criminal interrogations to detect deception. However, polygraphs can be difficult to administer under circumstances that prevent the use of biosensors. To address the shortcomings of the biosensors, deception-detection technology without biosensors is needed. We propose a deception-detection method, FacialCueNet, which is a multi-modal network that utilizes both facial images and facial cues based on deep-learning technology. FacialCueNet incorporates facial cues that indicate deception, such as action-unit frequency, symmetry, gaze pattern, and micro-expressions extracted from videos. Additionally, the spatial-temporal attention module, based on convolutional neural network and convolutional long short-term memory, is applied to FacialCueNet to provide interpretable information from interrogations. Because our goal was developing an algorithm applicable to criminal interrogations, we trained and evaluated FacialCueNet using the DDCIT dataset, which was collected using a data acquisition protocol similar to those used in actual investigations. To compare deception-detection performance with state-of-the-art works, a public dataset was also used. As a result, the mean deception-detection F1 score using the DDCIT dataset was 81.22%, with an accuracy of 70.79%, recall of 0.9476, and precision of 0.7107. When evaluating against the public database, our method demonstrated an evaluation accuracy of 88.45% and achieved an AUC of 0.9541, indicating a improvement of 1.25% compared to the previous results. We also present interpretive results of deception detection by analyzing the influence of spatial and temporal factors. These results show that FacialCueNet has the potential to detect deception using only facial videos. By providing interpretation of predictions, our system could be useful tool for criminal interrogation.

show abstract

Deception detection in speech using bark band and perceptually significant energy features

Cited by 15 publications

References 5 publications

Use of Machine Learning for Deception Detection From Spectral and Cepstral Features of Speech Signals

Use of Machine Learning for Deception Detection From Spectral and Cepstral Features of Speech Signals

Investigation of Speech Signal Parameters Reflecting the Truth of Transmitted Information

FacialCueNet: unmasking deception - an interpretable model for criminal interrogation using facial expressions

Contact Info

Product

Resources

About