Who’s the Best Charades Player? Mining Iconic Movement of Semantic Concepts

Hsieh, Yung-Huan; Hidayati, Shintami Chusnul; Cheng, Winton; Hu, Min‐Chun; Hua, Kai-Lung

doi:10.1007/978-3-319-04114-8_20

“…For example, the popular party game of charades requires one person (the 'actor') to act out a word or phrase without relying on speech or writing, while the other players (the 'guessers') attempt to guess that word or phrase (e.g. [66], see also [67]).…”

Section: (B) Charadesmentioning

confidence: 99%

Pantomimic fossils in modern human communication

Żywiczyński

¹

,

Wacewicz

²

,

Lister

³

2021

Phil. Trans. R. Soc. B

View full text Add to dashboard Cite

Bodily mimesis, the capacity to use the body representationally, was one of the key innovations that allowed early humans to go beyond the ‘baseline’ of generalized ape communication and cognition. We argue that the original human-specific communication afforded by bodily mimesis was based on signs that involve three entities: an expression that represents an object (i.e. communicated content) for an interpreter . We further propose that the core component of this communication, pantomime, was able to transmit referential information that was not limited to select semantic domains or the ‘here-and-now’, by means of motivated—most importantly iconic—signs. Pressures for expressivity and economy then led to conventionalization of signs and a growth of linguistic characteristics: semiotic systematicity and combinatorial expression. Despite these developments, both naturalistic and experimental data suggest that the system of pantomime did not disappear and is actively used by modern humans. Its contemporary manifestations, or pantomimic fossils , emerge when language cannot be used, for instance when people do not share a common language, or in situations where the use of (spoken) language is difficult, impossible or forbidden. Under such circumstances, people bootstrap communication by means of pantomime and, when these circumstances persist, newly emergent pantomimic communication becomes increasingly language-like. This article is part of the theme issue ‘Reconstructing prehistoric languages’.

show abstract

“…Finally, I determined that a study by Hsieh et al [6] is the most relevant to the present study, as they construed charades similarly as a natural form of data collection, "an acting game with the idea for one player to use physical (e.g. gesture or action) rather than verbal language to act out a semantic concept (i.e.…”

Section: Charades As a Modality Of Instruction To Collect Gesture Samplesmentioning

confidence: 99%

“…Consequently, each pose of It is important to note that this study did not intend to collect gesture samples to generate a 3D skeleton capable of providing specific GDP values: In contrast to human action recognition, the goal of our study is to evaluate the iconic level of a performing action video, i.e. whether it is vivid and expressive to describe the corresponding concept" [6]. In other words, Hsieh et al are specifically testing the viability of charades as a modality of instruction for gesture recognition research, and they have determined that it is suitable for conveying the patterns between semantic concepts and human actions.…”

Section: Charades As a Modality Of Instruction To Collect Gesture Samplesmentioning

confidence: 99%

See 1 more Smart Citation

Machine recognition of human gestures through principal joint variable analysis

Bulzacki¹

2021

Preprint

0

View full text Add to dashboard Cite

Gestures are a fundamental part of human communication and are becoming a key component of human-computer interaction. Traditionally, to teach computers to recognize specific gestures, researchers have used a sensor, usually a camera, to collect large gesture datasets, which are then classified and structured using machine learning techniques. Yet finding a way to confidently differentiate between several gesture classes has proven to be rather difficult for those working in the gesture recognition field. To capture the samples of movements necessary to train gesture recognition systems, the first step is to provide research participants with appropriate instructions. As collecting gesture data is the crucial first step of creating a robust gesture dataset, this dissertation will examine the modalities of instruction used in gesture recognition research to examine whether appropriate directives are conveyed to research participants. These experiments will result in the creation of a new dataset, the PJVA-20 dataset, comprised of 50 samples of 20 gesture classes sampled from 6 participants. After collecting the gesture samples of the PJVA-20 dataset, this dissertation will establish the benchmark recognition system PJVA — chiefly comprised of AMFE, Polynomial Motion Approximation, and Principal Component Analysis (PCA)—to contribute to the gesture recognition literature in terms of novel gesture recognition algorithms that can achieve high speed and accuracy results. This also involves examining studies in the gesture recognition literature to determine which machine learning algorithms offer reliability, speed, and accuracy for solving complex gesture recognition problems, as well as experimenting and testing the PJVA approach against other researchers in the Computer Vision and Machine Learning fields. In particular, the MSRC-12 research provides a benchmark point of comparison for research in this field. To test the quality of samples on the PJVA-20 against the MSRC-12, a new method is established for extracting motion feature vectors through a novel gesture recognition approach, AMFE. This is tested by applying PJVA to extract and label gesture data from both the MSRC- 12 and PJVA-20 datasets.

show abstract

“…This means that there must be a) appropriate examples of desired gestures (correctness) and b) the dataset must include common, desired variants of the particular movements associated with the gestures (coverage). Achieving both of these goals is important because the appropriate coverage ensures the resulting machine learning system is representative of the target population (in this case, as many body types as possible) while also determining the correct movements which "best match people's common stereotypes of the semantic concept associated with a gesture" [6].…”

Section: Thesis Structurementioning

confidence: 99%

Machine recognition of human gestures through principal joint variable analysis

Bulzacki¹

2021

Preprint

0

View full text Add to dashboard Cite

Gestures are a fundamental part of human communication and are becoming a key component of human-computer interaction. Traditionally, to teach computers to recognize specific gestures, researchers have used a sensor, usually a camera, to collect large gesture datasets, which are then classified and structured using machine learning techniques. Yet finding a way to confidently differentiate between several gesture classes has proven to be rather difficult for those working in the gesture recognition field. To capture the samples of movements necessary to train gesture recognition systems, the first step is to provide research participants with appropriate instructions. As collecting gesture data is the crucial first step of creating a robust gesture dataset, this dissertation will examine the modalities of instruction used in gesture recognition research to examine whether appropriate directives are conveyed to research participants. These experiments will result in the creation of a new dataset, the PJVA-20 dataset, comprised of 50 samples of 20 gesture classes sampled from 6 participants. After collecting the gesture samples of the PJVA-20 dataset, this dissertation will establish the benchmark recognition system PJVA — chiefly comprised of AMFE, Polynomial Motion Approximation, and Principal Component Analysis (PCA)—to contribute to the gesture recognition literature in terms of novel gesture recognition algorithms that can achieve high speed and accuracy results. This also involves examining studies in the gesture recognition literature to determine which machine learning algorithms offer reliability, speed, and accuracy for solving complex gesture recognition problems, as well as experimenting and testing the PJVA approach against other researchers in the Computer Vision and Machine Learning fields. In particular, the MSRC-12 research provides a benchmark point of comparison for research in this field. To test the quality of samples on the PJVA-20 against the MSRC-12, a new method is established for extracting motion feature vectors through a novel gesture recognition approach, AMFE. This is tested by applying PJVA to extract and label gesture data from both the MSRC- 12 and PJVA-20 datasets.

show abstract

Who’s the Best Charades Player? Mining Iconic Movement of Semantic Concepts

Cited by 8 publications

References 12 publications

Pantomimic fossils in modern human communication

Pantomimic fossils in modern human communication

Machine recognition of human gestures through principal joint variable analysis

Machine recognition of human gestures through principal joint variable analysis

Contact Info

Product

Resources

About