Action Recognition Online with Hierarchical Self-Organizing Maps

Gharaee, Zahra; Gärdenfors, Peter; Johnsson, Magnus

doi:10.1109/sitis.2016.91

Cited by 8 publications

(15 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the second experiment, the online experiment presented in Gharaee et al (2016) is developed by implementing the segmentation technique proposed in this study. Then, a dataset of actions composed of 3D joints postures is collected by using a Kinect sensor.…”

Section: Methodsmentioning

confidence: 99%

“…There are several other applications for action recognition systems including human-computer interaction, video retrieval, sign language recognition, medical health care, video analysis (sports video analysis), game industry and video surveillance. In earlier works (Gharaee et al 2016(Gharaee et al , 2017b, a, c; Gharaee 2018a), Gharaee et al developed a system for human action recognition using hierarchical architecture based on self-organizing neural networks. Recently this architecture is developed by using growing grid networks to improve its perfomance in recognizing human actions (Gharaee 2018b(Gharaee , 2020.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Online recognition of unsegmented actions with hierarchical SOM architecture

Gharaee

2020

Cogn Process

Self Cite

View full text Add to dashboard Cite

Automatic recognition of an online series of unsegmented actions requires a method for segmentation that determines when an action starts and when it ends. In this paper, a novel approach for recognizing unsegmented actions in online test experiments is proposed. The method uses self-organizing neural networks to build a three-layer cognitive architecture. The unique features of an action sequence are represented as a series of elicited key activations by the first-layer self-organizing map. An average length of a key activation vector is calculated for all action sequences in a training set and adjusted in learning trials to generate input patterns to the second-layer self-organizing map. The pattern vectors are clustered in the second layer, and the clusters are then labeled by an action identity in the third layer neural network. The experiment results show that although the performance drops slightly in online experiments compared to the offline tests, the ability of the proposed architecture to deal with the unsegmented action sequences as well as the online performance makes the system more plausible and practical in real-case scenarios.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Online recognition of unsegmented actions with hierarchical SOM architecture

Gharaee

2020

Cogn Process

Self Cite

View full text Add to dashboard Cite

show abstract

“…This can be accomplished by basically the same psychological methods used for investigating similarities between objects. In robotics, a common method for reducing the complexity of an action is to use a Kinect sensor that extracts the movements of a stick-figure representation of a person and then use a neural network to classify the reduced actions (e.g., References [43,44]). An analogy between how objects and how actions are represented in conceptual spaces is that action concepts share a similar structure with object categories; in particular, they have prototypes [45].…”

Section: Actionsmentioning

confidence: 99%

“…These systems are psychologically motived, and they have not been developed with the aim of supporting human-robot communication. Recent computational work by Gharaee and her colleagues has resulted in an online system based on neural networks that can categorize and segment a number of bodily actions [43,44,48]. Such a system can be used by a robot to select a relevant verb (see Section 4.3) to be used in the robot's construction of sentences describing what is happening.…”

Section: Actionsmentioning

confidence: 99%

“…In the implementation, only the actions of the iCub are considered, and they form the basis for the manner verbs that are used in the dialog. However, since the iCub can be combined with a Kinect camera, it is possible to extend the system to also allow actions performed by humans to be coded by using some action categorization system that works in real time (for example, Reference [43]). This would allow a richer communication between the iCub and humans.…”

Section: An Implementation For An Icub Robotmentioning

confidence: 99%

See 1 more Smart Citation

Using Event Representations to Generate Robot Semantics

Gärdenfors

2019

J. Hum.-Robot Interact.

View full text Add to dashboard Cite

Most semantic models employed in human-robot interactions concern how a robot can understand commands, but in this article the aim is to present a framework that allows dialogic interaction. The key idea is to use events as the fundamental structures for the semantic representations of a robot. Events are modeled in terms of conceptual spaces and mappings between spaces. It is shown how the semantics of major word classes can be described with the aid of conceptual spaces in a way that is amenable to computer implementations. An event is represented by two vectors, one force vector representing an action and one result vector representing the effect of the action. The two-vector model is then extended by the thematic roles so an event is built up from an agent, an action, a patient, and a result. It is shown how the components of an event can be put together to semantic structures that represent the meanings of sentences. It is argued that a semantic framework based on events can generate a general representational framework for humanrobot communication. An implementation of the framework involving communication with an iCub will be described.

show abstract

First and second order dynamics in a hierarchical SOM system for action recognition

Gharaee

Gärdenfors

Johnsson

2017

Applied Soft Computing

Self Cite

View full text Add to dashboard Cite

Human recognition of the actions of other humans is very efficient and is based on patterns of movements. Our theoretical starting point is that the dynamics of the joint movements is important to action categorization. On the basis of this theory, we present a novel action recognition system that employs a hierarchy of Self-Organizing Maps together with a custom supervised neural network that learns to categorize actions. The system preprocesses the input from a Kinect like 3D camera to exploit the information not only about joint positions, but also their first and second order dynamics. We evaluate our system in two experiments with publicly available data sets, and compare its performance to the performance with less sophisticated preprocessing of the input. The results show that including the dynamics of the actions improves the performance. We also apply an attention mechanism that focuses on the parts of the body that are the most involved in performing the actions.

show abstract

Action Recognition Online with Hierarchical Self-Organizing Maps

Cited by 8 publications

References 26 publications

Online recognition of unsegmented actions with hierarchical SOM architecture

Online recognition of unsegmented actions with hierarchical SOM architecture

Using Event Representations to Generate Robot Semantics

First and second order dynamics in a hierarchical SOM system for action recognition

Contact Info

Product

Resources

About