Classification of patient case discussions through analysis of vocalisation graphs

Abstract

This paper investigates the use of amount and structure of talk as a basis for automatic classification of patient case discussions in multidisciplinary medical team meetings recorded in a real-world setting. We model patient case discussions as vocalisation graphs, building on research from the fields of interaction analysis and social psychology. These graphs are "content free" in that they only encode patterns of vocalisation and silence. The fact that it does not rely on automatic transcription makes the technique presented in this paper an attractive complement to more sophisticated speech processing methods as a means of indexing medical team meetings. We show that despite the simplicity of the underlying representation mechanism, accurate classification performance (F-scores: F1 = 0.98, for medical patient case discussions, and F1 = 0.97, for surgical case discussions) can be achieved with a simple k-nearest neighbour classifier when vocalisations are represented at the level of individual speakers. Possible applications of the method in health informatics for storage and retrieval of multimedia medical meeting records are discussed.

Automatic identification of experts and performance prediction in the multimodal math data corpus through analysis of speech interaction

“…Second, we make no use of prosodic features (speech rate, pitch, loudness) other than pause duration, though we discuss briefly how these features could be incorporated into our data representation scheme in future work (section 6). Third, we structure speech features as a vocalisation graph [5,13] rather than consider them in isolation.…”

“…In previous research, representations based on vocalisation graphs have been successfully used, though in quite different ways, to support qualitative analyses of group behaviour [5] and clinical dialogue [7], and, in computer science, to automatically segment medical team meetings [12] and categorise patient case discussion sessions [13]. In the work reported here, we modify the underlying representation employed in those works so as to abstract away speaker information.…”

Models and operators for extension of active multimedia documents via annotations

“…In the context of intra-media browsing, a browsing tool can be focused on exploration of a single media element or of multiple media elements. Browsing a single media element encompasses for instance exploring a single video [Schoeffmann and Boeszoermenyi, 2009] or the timeline of an audio recording [Luz and Kane, 2009]. Multiple media elements are more relevant when browsing synchronized multimedia presentations: in this case, multiple media elements are orchestrated in a way that reconstruct their captured time relationships.…”