Audio-Assisted Movie Dialogue Detection

Abstract

Abstract-An audio-assisted system is investigated that detects if a movie scene is a dialogue or not. The system is based on actor indicator functions. That is, functions which define if an actor speaks at a certain time instant. In particular, the crosscorrelation and the magnitude of the corresponding the crosspower spectral density of a pair of indicator functions are input to various classifiers, such as voted perceptrons, radial basis function networks, random trees, and support vector machines for dialogue/non-dialogue detection. To boost classifier efficiency AdaBoost is also exploited. The aforementioned classifiers are trained using ground truth indicator functions determined by human annotators for 41 dialogue and another 20 non-dialogue audio instances. For testing, actual indicator functions are derived by applying audio activity detection and actor clustering to audio recordings. 23 instances are randomly chosen among the aforementioned 41 dialogue instances, 17 of which correspond to dialogue scenes and 6 to non-dialogue ones. Accuracy ranging between 0.739 and 0.826 is reported.

An acoustic scene classification approach involving domestic violence using machine learning

“…Observamos que o parâmetro de áudio MFCC demonstrou-se ser superior na coleta de informações do áudio para identificação de cenas de violência doméstica em comparação com os outros parâmetros ZCR e Energia, alcançando uma acurácia média de 73,14%. Além disso, pelo fato de até o momento dessa pesquisa não termos encontrados em nossas nenhum estudo dessa natureza sendo realizado, abordando especificamente cenas de violência doméstica, consideramos que nossa abordagem apresenta bons resultados se comparados com outros trabalhos envolvendo apenas reconhecimento de algum outro tipo de cena acústica, utilizando parâmetros de áudios, e que obtiveram resultados de acurácias não tão distantes dos nossos, como por exemplo em [Kotti et al 2008] com 82%, [Stowell et al 2015] apresentando sistemas com acurácias entre 55% e 77% , [Yang et al 2016] com 79,9%, [Elizalde et al 2016] Para trabalhos futuros, poderemos utilizar uma abordagem de utilização de diferentes classificadores utilizando os parâmetros MFCC e Energia isoladamente e em conjunto para o aprendizado de máquina, pois demonstraram ter melhores resultados do que ZCR, considerando a ampliação do número de amostras de áudios em nossa base de dados, pois acreditamos que essa evolução da base pode ampliar a capacidade de aprendizado de máquina e consequentemente alcançar uma melhor acurácia como resultado.…”

Audio Event Change Detection and Clustering in Movies

“…Researchers have found that audio may play more important than video for the content analysis of audiovideo sometimes [18], and audio information processing is much faster than video information processing [19]. Hence,audio information is used in film content analysis and understanding more and more in recent years.…”

“…In film content analysis and understanding, audio event change detection is the most important step, which is base of movie audio content analysis and understanding. In past, there are some film audio processing reports which focus on the detection people and music, for example, University of Southern California have studied two people dialogue, many people dialogue and mixture events in movies [5], Aristotle University of Thessaloniki used audio detect movie dialogue [19]. Asian Institute of Technology of Thailand classified movies into several kinds by detection music and song in movies.…”

Generative modeling and classification of dialogs by a low-level turn-taking feature

“…An important aim of an external mediator is to obtain a good yet general blueprint of a dialog situation by analyzing the ongoing conversational dynamics, intended as the alternating speech behavior exploited by the partners during negotiation [2,4,3]. The ability to carefully capture and classify conversational dynamics could also be employed to enhance the performance of a wide range of applications, such as dialog detection [9], speaker recognition/verification [5], and event detection in meeting scenarios [10], also considering video cues. More consistently, it would improve social signalling applications [1][2][3][11][12][13][14], such as the ones that link conversational dynamics to social roles (e.g., dominance [15], mirroring [3] and others [1]), or those that face interesting and complex challenges such as the ''thin slice'' detection, i.e., the ability of predicting the outcome of a specific conversational exchange in very limited time [13].…”