Semantic Recognition of Human-Object Interactions via Gaussian-Based Elliptical Modeling and Pixel-Level Labeling

Abstract

Human-Object Interaction (HOI) recognition, due to its significance in many computer visionbased applications, requires in-depth and meaningful details from image sequences. Incorporating semantics in scene understanding has led to a deep understanding of human-centric actions. Therefore, in this research work, we propose a semantic HOI recognition system based on multi-vision sensors. In the proposed system, the de-noised RGB and depth images, via Bilateral Filtering (BLF), are segmented into multiple clusters using a Simple Linear Iterative Clustering (SLIC) algorithm. The skeleton is then extracted from segmented RGB and depth images via Euclidean Distance Transform (EDT). Human joints, extracted from the skeleton, provide the annotations for accurate pixel-level labeling. An elliptical human model is then generated via a Gaussian Mixture Model (GMM). A Conditional Random Field (CRF) model is trained to allocate a specific label to each pixel of different human body parts and an interaction object. Two semantic feature types that are extracted from each labeled body part of the human and labelled objects are: Fiducial points and 3D point cloud. Features descriptors are quantized using Fisher's Linear Discriminant Analysis (FLDA) and classified using K-ary Tree Hashing (KATH). In experimentation phase the recognition accuracy achieved with the Sports dataset is 92.88%, with the Sun Yat-Sen University (SYSU) 3D HOI dataset is 93.5% and with the Nanyang Technological University (NTU) RGB+D dataset it is 94.16%. The proposed system is validated via extensive experimentation and should be applicable to many computer-vision based applications such as healthcare monitoring, security systems and assisted living etc.INDEX TERMS 3D point cloud, fiducial points, human-object interaction, pixel labeling, semantic segmentation, super-pixels, K-ary tree hashing.

Physical human locomotion prediction using manifold regularization

“…Random fields is the general form of stochastic process where we don’t need the real values and it can take multi-dimensional matrix or points Khalid et al (2021) . Hence, we applied it over multi-dimensional signals like motion and physiological signals together.…”

“…Next, to decide the kinematic-static patterns among the windows ( Al Shloul et al, 2022 ), we suggested a polynomial probability distribution. For kinematic patterned signal data, multisynchrosqueezing transform ( Yu, Wang & Zhao, 2019 ) and hidden Markov random field ( Wang, 2012 ) are suggested for features extraction, whereas, for static patterned signal windows, dynamic time warping ( Laperre, Amaya & Lapenta, 2020 ) and Gaussian Markov random field ( Khalid et al, 2021 ) are recommended. Then, two feature optimization techniques are used including quadratic discriminant analysis and orthogonal fuzzy neighborhood discriminant analysis.…”

Sensors-Based Ambient Assistant Living via E-Monitoring Technology

“…Then, the convolutional neural networks (ConvNets) and Polynormal fisher vector was used for classification. Another system [16] denoised depth images through Bilateral Filter (BLF) and segmented image clusters using Simple Linear Iterative Clustering (SLIC) algorithm. The actions were then classified using K-ary Tree Hashing (KTH).…”

Nonlinear Random Matrix Model and Research for Quantitative Representation of Volleyball Attacker’s Action Links

“…Domestic researchers mainly focus on the comparison of women's volleyball team's offensive ability with their opponents in different world competitions, especially statistics on offensive indicators such as serving, passing, and attacking. It is one of the important standards of the world's strong teams, and the women's volleyball team needs to strengthen the training of the stability of the attack and smash [9,10].…”