PRAXIS: Towards automatic cognitive assessment using gesture recognition

Negin, Farhood; Rodríguez, Paul; Koperski, Michal; Kerboua, Adlen; Gonzàlez, Jordi; Bourgeois, J.; Chapoulie, Emmanuelle; Pierpoint, Robert; Brémond, François

doi:10.1016/j.eswa.2018.03.063

Cited by 34 publications

(39 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This subtle detail is a requirement of our proposed system. Progress in recognising fine-grained actions in videos has been made by explicitly considering 'high-level' features relating to the human within the images (Kläser et al, 2010;Yao et al, 2010;Chéron et al, 2015;Choutas et al, 2018;Luvizon et al, 2018;Negin et al, 2018). An example of this is performing automatic human pose estimation in order to extract appearance-based features from specific body parts, rather than across the entire image (Chéron et al, 2015).…”

Section: Human Action Recognitionmentioning

confidence: 99%

The development of a video retrieval system using a clinician-led approach

Darby

Sánchez

Bew

et al. 2020

Expert Systems with Applications

View full text Add to dashboard Cite

Patient video taken at home can provide valuable insights into the recovery progress during a programme of physical therapy, but is very time consuming for clinician review. Our work focussed on i) enabling any patient to share information about progress at home, simply by sharing video and ii) building intelligent systems to support Physical Therapists (PTs) in reviewing this video data and extracting the necessary detail. This paper reports the development of the system, appropriate for future clinical use without reliance on a technical team, and the clinician involvement in that development. We contribute an interactive contentbased video retrieval system that significantly reduces the time taken for clinicians to review videos, using human head movement as an example. The system supports query-by-movement (clinicians move their own body to define search queries) and retrieves the essential fine-grained movements needed for clinical interpretation. This is done by comparing sequences of image-based pose estimates (here head rotations) through a distance metric (here Fréchet distance) and presenting a ranked list of similar movements to clinicians for review. In contrast to existing intelligent systems for retrospective review of human movement, the system supports a flexible analysis where clinicians can look for any movement that interests them. Evaluation by a group of PTs with expertise in training movement control showed that 96% of all relevant movements were identified with time savings of as much as 99.1% compared to reviewing target videos in full. The novelty of this contribution includes retrospective progress monitoring that preserves context through video, and content-based video retrieval that supports both fine-grained human actions and queryby-movement. Future research, including large clinician-led studies, will refine the technical aspects and explore the benefits in terms of patient outcomes, PT time, and financial savings over the course of a programme of therapy. It is anticipated that this clinician-led approach will mitigate the reported slow clinical uptake of technology with resulting patient benefit.

show abstract

Section: Human Action Recognitionmentioning

confidence: 99%

The development of a video retrieval system using a clinician-led approach

Darby

Sánchez

Bew

et al. 2020

Expert Systems with Applications

View full text Add to dashboard Cite

show abstract

“…Negin ey al. [21] introduced a multifunctional perceptron for sign language recognition and synthesis; the main functions include realtime recognition of large vocabulary of sign language, a synthesis system for sign language in personal computer (PC) and tablet, and face image monitoring and recognition. Shahzad and Zhang [22] collected gesture information with two pairs of data gloves and three position sensors, and recognized more than 93% of 280 gesture models, using Gaussian mixture model and hidden Markov model.…”

Section: Introductionmentioning

confidence: 99%

Gesture Feature Extraction and Recognition Based on Image Processing

Wang¹,

Yang²,

Zhang³

2020

View full text Add to dashboard Cite

Gesture recognition has become increasingly popular, in response to the growing demand for intelligent and personalized human-computer interaction (HCI) and human-to-human interaction. However, gesture recognition raises a high requirement on the background color of the gesture image, and faces difficulty in extracting multiple gesture features. To solve these problems, this paper presents a novel approach for gesture feature extraction and recognition based on image processing. Firstly, the workflow of the proposed gesture recognition method was given, and a series of preprocessing was performed on the original gesture image, prior to formal extraction and recognition. Next, the authors detailed the extraction of features from gesture boundaries and fingertips. Finally, a convolutional neural network (CNN) was constructed for gesture recognition, and a gesture recognition model was developed based on residual network. The proposed approach was proved to be valid through experiments. The research results provide a reference for the application of CNN in the recognition of various postures or shapes.

show abstract

“…Another example is traffic police officers who direct traffic during a traffic jam or car accident. Some researchers have also used hand gesture recognition to detect whether a person has Alzheimer's disease [7] or to interpret hand-drawn diagrams [8]. In musical performances, some use the Kinect to recognize drum-hitting gestures [9].…”

Section: Introductionmentioning

confidence: 99%

Real-Time Musical Conducting Gesture Recognition Based on a Dynamic Time Warping Classifier Using a Single-Depth Camera

Chin-Shyurng

Lee

2019

Applied Sciences

View full text Add to dashboard Cite

Gesture recognition is a human−computer interaction method, which is widely used for educational, medical, and entertainment purposes. Humans also use gestures to communicate with each other, and musical conducting uses gestures in this way. In musical conducting, conductors wave their hands to control the speed and strength of the music played. However, beginners may have a limited comprehension of the gestures and might not be able to properly follow the ensembles. Therefore, this paper proposes a real-time musical conducting gesture recognition system to help music players improve their performance. We used a single-depth camera to capture image inputs and establish a real-time dynamic gesture recognition system. The Kinect software development kit created a skeleton model by capturing the palm position. Different palm gestures were collected to develop training templates for musical conducting. The dynamic time warping algorithm was applied to recognize the different conducting gestures at various conducting speeds, thereby achieving real-time dynamic musical conducting gesture recognition. In the experiment, we used 5600 examples of three basic types of musical conducting gestures, including seven capturing angles and five performing speeds for evaluation. The experimental result showed that the average accuracy was 89.17% in 30 frames per second.

show abstract

PRAXIS: Towards automatic cognitive assessment using gesture recognition

Cited by 34 publications

References 69 publications

The development of a video retrieval system using a clinician-led approach

The development of a video retrieval system using a clinician-led approach

Gesture Feature Extraction and Recognition Based on Image Processing

Real-Time Musical Conducting Gesture Recognition Based on a Dynamic Time Warping Classifier Using a Single-Depth Camera

Contact Info

Product

Resources

About