Human-Robot Interaction with Smart Shopping Trolley Using Sign Language: Data Collection

Ryumin, Dmitry; Ivanko, Denis; Axyonov, Alexandr; Kagirov, Ildar; Karpov, Alexey; Železný, Miloš

doi:10.1109/percomw.2019.8730886

Cited by 13 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Or second, if we need to recognize continuous speech (phrases or sentences). Based on this information we have to choose different size of a database, different methods for features extraction and model training (Ryumin et al, 2019).…”

Section: Backgrounds and Related Researchmentioning

confidence: 99%

A Novel Task-Oriented Approach Toward Automated Lip-Reading System Implementation

Ivanko

Ryumin

2021

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. Visual information plays a key role in automatic speech recognition (ASR) when audio is corrupted by background noise, or even inaccessible. Speech recognition using visual information is called lip-reading. The initial idea of visual speech recognition comes from humans’ experience: we are able to recognize spoken words from the observation of a speaker's face without or with limited access to the sound part of the voice. Based on the conducted experimental evaluations as well as on analysis of the research field we propose a novel task-oriented approach towards practical lip-reading system implementation. Its main purpose is to be some kind of a roadmap for researchers who need to build a reliable visual speech recognition system for their task. In a rough approximation, we can divide the task of lip-reading into two parts, depending on the complexity of the problem. First, if we need to recognize isolated words, numbers or small phrases (e.g. Telephone numbers with a strict grammar or keywords). Or second, if we need to recognize continuous speech (phrases or sentences). All these stages disclosed in detail in this paper. Based on the proposed approach we implemented from scratch automatic visual speech recognition systems of three different architectures: GMM-CHMM, DNN-HMM and purely End-to-end. A description of the methodology, tools, step-by-step development and all necessary parameters are disclosed in detail in current paper. It is worth noting that for the Russian speech recognition, such systems were created for the first time.

show abstract

Section: Backgrounds and Related Researchmentioning

confidence: 99%

A Novel Task-Oriented Approach Toward Automated Lip-Reading System Implementation

Ivanko

Ryumin

2021

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…This database was collected using the developed automatic 3D video stream recording system with the Microsoft Kinect 2.0 rangefinder sensor. The overall architecture of the developed system (MulGesRecDB) is presented in Figure 2 (Ryumin et al, 2019).…”

Section: Datasetmentioning

confidence: 99%

Automatic Detection and Recognition of 3d Manual Gestures for Human-Machine Interaction

Ryumin

Kagirov

Ivanko

et al. 2019

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

<p><strong>Abstract.</strong> In this paper, we propose an approach to detect and recognize 3D one-handed gestures for human-machine interaction. The logical structure of the modules of the system for recording a gestural database is described. The logical structure of the database of 3D gestures is presented. Examples of frames showing gestures in the format of Full High Definition, in the map depth mode and in the infrared illustrated. Models of a deep convolutional network for detecting faces and hand shapes are described. The results of automatic detection of the area with the face and the shape of the hand are given. Identified the distinctive features of the gesture at a certain point in time. The process of recognizing 3D one-handed gestures is described. Due to its versatility, this method can be used in tasks of biometrics, computer vision, machine learning, automatic systems of face recognition, sign languages.</p>

show abstract

“…Computer vision and machine learning are the main means how to process such data. They can extend the devices and applications with intelligent algorithms that understand what users are doing and how they interact with their surroundings [1]. In general, hand pose estimation is being addressed in many fields -robotics [2], medicine, automotive, or sign language processing [3].…”

Section: Introductionmentioning

confidence: 99%

Hand Pose Estimation in the Task of Egocentric Actions

2021

View full text Add to dashboard Cite

Human-Robot Interaction with Smart Shopping Trolley Using Sign Language: Data Collection

Cited by 13 publications

References 18 publications

A Novel Task-Oriented Approach Toward Automated Lip-Reading System Implementation

A Novel Task-Oriented Approach Toward Automated Lip-Reading System Implementation

Automatic Detection and Recognition of 3d Manual Gestures for Human-Machine Interaction

Hand Pose Estimation in the Task of Egocentric Actions

Contact Info

Product

Resources

About