Modalities Combination for Italian Sign Language Extraction and Recognition

Seddik, Bassem; Gazzah, Sami; Amara, Najoua Essoukri Ben

doi:10.1007/978-3-319-23234-8_65

Cited by 6 publications

(9 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Human poses are important cues for video analysis in a variety of tasks such as activity/action recognition [9,10], multi-object detection [11], and sign language processing and recognition [12]. Generally, HPE approaches can be divided into two main groups: traditional HPE approaches and deep learning-based ones.…”

Section: Related Workmentioning

confidence: 99%

Deep Full-Body HPE for Activity Recognition from RGB Frames Only

Boualia

Amara

2021

Informatics

View full text Add to dashboard Cite

Human Pose Estimation (HPE) is defined as the problem of human joints’ localization (also known as keypoints: elbows, wrists, etc.) in images or videos. It is also defined as the search for a specific pose in space of all articulated joints. HPE has recently received significant attention from the scientific community. The main reason behind this trend is that pose estimation is considered as a key step for many computer vision tasks. Although many approaches have reported promising results, this domain remains largely unsolved due to several challenges such as occlusions, small and barely visible joints, and variations in clothing and lighting. In the last few years, the power of deep neural networks has been demonstrated in a wide variety of computer vision problems and especially the HPE task. In this context, we present in this paper a Deep Full-Body-HPE (DFB-HPE) approach from RGB images only. Based on ConvNets, fifteen human joint positions are predicted and can be further exploited for a large range of applications such as gesture recognition, sports performance analysis, or human-robot interaction. To evaluate the proposed deep pose estimation model, we apply it to recognize the daily activities of a person in an unconstrained environment. Therefore, the extracted features, represented by deep estimated poses, are fed to an SVM classifier. To validate the proposed architecture, our approach is tested on two publicly available benchmarks for pose estimation and activity recognition, namely the J-HMDBand CAD-60datasets. The obtained results demonstrate the efficiency of the proposed method based on ConvNets and SVM and prove how deep pose estimation can improve the recognition accuracy. By means of comparison with state-of-the-art methods, we achieve the best HPE performance, as well as the best activity recognition precision on the CAD-60 dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Full-Body HPE for Activity Recognition from RGB Frames Only

Boualia

Amara

2021

Informatics

View full text Add to dashboard Cite

show abstract

“…Liang et al [161] use rather the joint modality to implement a similar scheme and decide on the dominant hand in motion. The CD approach presented by Seddik et al [159] uses the joint inputs to segment both the streams and the learning population into right, left and bihanded actions (see Fig. 1.9), thus allowing classifiers specialization in later steps.…”

Section: Unimodal Temporal Segmentation Approachesmentioning

confidence: 99%

“…Fig. 1.9 Example temporal segmentation results: (a) and (b) Left and right hands motion curves, (c) TS results and (d) the ground truth[159] …”

mentioning

confidence: 99%

Visual Methods for Sign Language Recognition: A Modality-Based Review

Seddik¹,

Amara²

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Sign language visual recognition from continuous multi-modal streams is still one of the most challenging fields. Recent advances in human actions recognition are exploiting the ascension of GPU-based learning from massive data, and are getting closer to human-like performances. They are then prone to creating interactive services for the deaf and hearing-impaired communities. A population that is expected to grow considerably in the years to come. This paper aims at reviewing the human actions recognition literature with the sign-language visual understanding as a scope. The methods analyzed will be mainly organized according to the different types of unimodal inputs exploited, their relative multi-modal combinations and pipeline steps. In each section, we will detail and compare the related datasets, approaches then distinguish the still open contribution paths suitable for the creation of sign language related services. Special attention will be paid to the approaches and commercial solutions handling facial expressions and continuous signing.

show abstract

“…We exploit here the joint‐relative descriptors efficiency and the sparse representations high linear separability. We extend our previous research [20, 21] with what follows: (i) compared to [20], we improve the joint normalisation, the RGB and depth feature binning and use richer BoVW representations. Furthermore, (ii) we evaluate the local–global approach of [21] within multiple fusion configurations using a variety of features concatenations.…”

Section: Introductionmentioning

confidence: 99%

Human‐action recognition using a multi‐layered fusion scheme of Kinect modalities

Seddik

Gazzah

Amara

2017

IET Computer Vision

Self Cite

View full text Add to dashboard Cite

This study addresses the problem of efficiently combining the joint, RGB and depth modalities of the Kinect sensor in order to recognise human actions. For this purpose, a multi-layered fusion scheme concatenates different specific features, builds specialised local and global SVM models and then iteratively fuses their different scores. The authors essentially contribute in two levels: (i) they combine the performance of local descriptors with the strength of global bags-of-visual-words representations. They are able then to generate improved local decisions that allow noisy frames handling. (ii) They also study the performance of multiple fusion schemes guided by different features concatenations, Fisher vectors representations concatenation and later iterative scores fusion. To prove the efficiency of their approach, they have evaluated their experiments on two challenging public datasets: CAD-60 and CGC-2014. Competitive results are obtained for both benchmarks.

show abstract

Modalities Combination for Italian Sign Language Extraction and Recognition

Cited by 6 publications

References 24 publications

Deep Full-Body HPE for Activity Recognition from RGB Frames Only

Deep Full-Body HPE for Activity Recognition from RGB Frames Only

Visual Methods for Sign Language Recognition: A Modality-Based Review

Human‐action recognition using a multi‐layered fusion scheme of Kinect modalities

Contact Info

Product

Resources

About