Online recognition of unsegmented actions with hierarchical SOM architecture

Gharaee, Zahra

doi:10.1007/s10339-020-00986-4

Cited by 6 publications

(3 citation statements)

References 51 publications

(129 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SOM is an unsupervised machine learning technique that produces the low-dimensional representation of data. In the literature, SOM is used is used for action recognition (Gharaee, 2021;Huang & Wu, 2010), classification (Wünstel, Polani, Uthmann, & Perl, 2000) and analysis (Bauer & Schöllhorn, 1997). Moreover, compared to t-SNE and PCA, SOM preserves topological relationships in the data (Dou, Xu, Shen, & Zhao, 2021), which makes it suitable for the visualization Figure 7.…”

Section: Resultsmentioning

confidence: 99%

Predictive Event Segmentation and Representation with Neural Networks: A Self-Supervised Model Assessed by Psychological Experiments

Basgol¹,

Ayhan²,

Uğur³

2022

Preprint

View full text Add to dashboard Cite

People segment complex, ever-changing and continuous experience into basic, stable and discrete spatio-temporal experience units, called events. Event segmentation literature investigates the mechanisms that allow people to extract these units from the continuous experience.Aiming to shed light on event segmentation ability, event segmentation theory points out that people predict ongoing activities and observe prediction error signals in order to find event boundaries that keep events apart. In this study, we investigated the mechanism giving rise to this ability by a computational model and accompanying psychological experiments. Inspired from the principles of event segmentation theory and predictive processing, we introduced a semi-mechanistic model of event segmentation, learning, and representation. This model consists of feed-forward neural networks that predict the sensory signal in the next time-step in order to represent different events, and a cognitive model that regulates these neural networks on the basis of their prediction errors. In order to verify the ability of our model in segmenting experience into spatio-temporal units, learning them during passive observation, and representing them in its internal representational space, we prepared a video that depicts natural human behaviors represented by point-light displays. We compared event segmentation behaviors of human participants and our model with this video in two hierarchical event segmentation levels.By using point-biserial correlation technique, we demonstrated that event segmentation decisions of our model correlated with the responses of participants. Moreover, by approximating internal representation space of participants by a similarity-based technique, we showed that our model formed a similar internal representation space with those of participants. Our results suggests that our model that tracks the prediction error signals can produce human-like event segmentation decisions and event representations. Finally, we discussed our contribution to the literature of event cognition and our understanding of how event segmentation is implemented in the brain.

show abstract

Section: Resultsmentioning

confidence: 99%

Predictive Event Segmentation and Representation with Neural Networks: A Self-Supervised Model Assessed by Psychological Experiments

Basgol¹,

Ayhan²,

Uğur³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Running several experiments using three different datasets of actions show (Wan, 2015) and Florence is the Florence3DAction dataset by (Seidenari et al, 2013) Internal simulation (%) the accuracy of the proposed framework in recognizing and predicting the intended actions. In future, developing the architecture applying information of prediction and objects involved in action to improve segmention of a sequence recognized online (Gharaee et al, 2016;Gharaee, 2020b) is crucial.…”

Section: Discussionmentioning

confidence: 99%

Predicting the intended action using internal simulation of perception

Gharaee¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

This article proposes an architecture, which allows the prediction of intention by internally simulating perceptual states represented by action pattern vectors. To this end, associative self-organising neural networks (A-SOM) is utilised to build a hierarchical cognitive archi-tecture for recognition and simulation of the skeleton based human actions. The abilities of the proposed architecture in recognising and predicting actions is evaluated in experiments using three different datasets of 3D actions. Based on the experiments of this article, apply-ing internally simulated perceptual states represented by action pattern vectors improves the performance of the recognition task in all experiments. Furthermore, internal simulation of perception addresses the problem of having limited access to the sensory input, and also the future prediction of the consecutive perceptual sequences. The performance of the system is compared and discussed with similar architecture using self-organizing neural networks (SOM).

show abstract

“…The number of regions represents basic directional information (three vertical divisions) and distance information (two horizontal divisions) of the scene [19]. This approach can facilitate further studies of how agent learns to control its attention to each region and if an attention mechanism could improve the performance of the task [21], [22]. To further reduce the dimensionality of the input, the semantic labels are clustered into five categories as: Road, Road-line, Off-road, Static object and Dynamic object.…”

Section: B Reinforcement Learning Methods For Vehicular Controlmentioning

confidence: 99%

A Bayesian Approach to Reinforcement Learning of Vision-Based Vehicular Control

Gharaee¹,

Holmquist²,

He³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we present a state-of-the-art reinforcement learning method for autonomous driving. Our approach employs temporal difference learning in a Bayesian framework to learn vehicle control signals from sensor data. The agent has access to images from a forward facing camera, which are preprocessed to generate semantic segmentation maps. We trained our system using both ground truth and estimated semantic segmentation input.Based on our observations from a large set of experiments, we conclude that training the system on ground truth input data leads to better performance than training the system on estimated input even if estimated input is used for evaluation.The system is trained and evaluated in a realistic simulated urban environment using the CARLA simulator. The simulator also contains a benchmark that allows for comparing to other systems and methods. The required training time of the system is shown to be lower and the performance on the benchmark superior to competing approaches.

show abstract

Online recognition of unsegmented actions with hierarchical SOM architecture

Cited by 6 publications

References 51 publications

Predictive Event Segmentation and Representation with Neural Networks: A Self-Supervised Model Assessed by Psychological Experiments

Predictive Event Segmentation and Representation with Neural Networks: A Self-Supervised Model Assessed by Psychological Experiments

Predicting the intended action using internal simulation of perception

A Bayesian Approach to Reinforcement Learning of Vision-Based Vehicular Control

Contact Info

Product

Resources

About