Automatic annotation of tennis games: An integration of audio, vision, and learning

Yan, Fei; Kittler, Josef; Windridge, David; Christmas, William J.; Mikolajczyk, Krystian; Cox, Stephen J.; Huang, Qiang

doi:10.1016/j.imavis.2014.08.004

Cited by 26 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, the use of domain experts’ to manually label data typically results in more truthful labeling, but it can be high-priced and time-consuming. In contrast, the use of fully automated labeling mechanisms can reduce time but may not be as precise as those delivered by a domain expert [ 20 , 21 , 22 ]. This study challenges the online and self-labeling scenarios in a realistic setting.…”

Section: Related Workmentioning

confidence: 99%

On-Device Deep Personalization for Robust Activity Data Collection

Mairittha

Inoue

2020

Sensors

View full text Add to dashboard Cite

One of the biggest challenges of activity data collection is the need to rely on users and keep them engaged to continually provide labels. Recent breakthroughs in mobile platforms have proven effective in bringing deep neural networks powered intelligence into mobile devices. This study proposes a novel on-device personalization for data labeling for an activity recognition system using mobile sensing. The key idea behind this system is that estimated activities personalized for a specific individual user can be used as feedback to motivate user contribution and improve data labeling quality. First, we exploited fine-tuning using a Deep Recurrent Neural Network to address the lack of sufficient training data and minimize the need for training deep learning on mobile devices from scratch. Second, we utilized a model pruning technique to reduce the computation cost of on-device personalization without affecting the accuracy. Finally, we built a robust activity data labeling system by integrating the two techniques outlined above, allowing the mobile application to create a personalized experience for the user. To demonstrate the proposed model’s capability and feasibility, we developed and deployed the proposed system to realistic settings. For our experimental setup, we gathered more than 16,800 activity windows from 12 activity classes using smartphone sensors. We empirically evaluated the proposed quality by comparing it with a baseline using machine learning. Our results indicate that the proposed system effectively improved activity accuracy recognition for individual users and reduced cost and latency for inference for mobile devices. Based on our findings, we highlight critical and promising future research directions regarding the design of efficient activity data collection with on-device personalization.

show abstract

Section: Related Workmentioning

confidence: 99%

On-Device Deep Personalization for Robust Activity Data Collection

Mairittha

Inoue

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Automating the segmentation and labeling stages of annotation requires computational techniques to identify the segments in which the data of interest is contained [22], and a knowledge the database that contains all the possible data to be labeled. The automated annotation approach presented as a system in [23] integrates computer vision, audio processing and machine learning to produce automated annotations of tennis game videos. The work presented by [24] proposes the use of a number of automatic detection components to create layers of annotations that can be integrated in tools like ELAN, supporting the work of human annotators.…”

Section: Approaches To Data Labelingmentioning

confidence: 99%

Semi-Automated Data Labeling for Activity Recognition in Pervasive Healthcare

Cruz-Sandoval

Beltrán-Márquez

García-Constantino

et al. 2019

Sensors

View full text Add to dashboard Cite

Activity recognition, a key component in pervasive healthcare monitoring, relies on classification algorithms that require labeled data of individuals performing the activity of interest to train accurate models. Labeling data can be performed in a lab setting where an individual enacts the activity under controlled conditions. The ubiquity of mobile and wearable sensors allows the collection of large datasets from individuals performing activities in naturalistic conditions. Gathering accurate data labels for activity recognition is typically an expensive and time-consuming process. In this paper we present two novel approaches for semi-automated online data labeling performed by the individual executing the activity of interest. The approaches have been designed to address two of the limitations of self-annotation: (i) The burden on the user performing and annotating the activity, and (ii) the lack of accuracy due to the user labeling the data minutes or hours after the completion of an activity. The first approach is based on the recognition of subtle finger gestures performed in response to a data-labeling query. The second approach focuses on labeling activities that have an auditory manifestation and uses a classifier to have an initial estimation of the activity, and a conversational agent to ask the participant for clarification or for additional data. Both approaches are described, evaluated in controlled experiments to assess their feasibility and their advantages and limitations are discussed. Results show that while both studies have limitations, they achieve 80% to 90% precision.

show abstract

“…Large variations of player appearances and partial occlusion are common issues not only for badminton but also for other racket sports, especially tennis, which has attracted much attention worldwide 11 improve the observation quality by removing occluders.…”

Section: Related Workmentioning

confidence: 99%

[Paper] Improvement of Badminton-Player Tracking Applying Image Pixel Compensation

Kamiyama

Kameda

Ohta

et al. 2017

MTA

View full text Add to dashboard Cite

Since the motion analysis of athletes is expected to provide critical information for improving training and strategy meetings, visual player-tracking techniques are being researched and developed 1)-4) . Since sporting matches are usually held over a long time period, the analysis time and effort can be drastically increased. On the other hand, however, assistant coaches and players want the analysis data as quickly as possible. Thus, high-speed automatic processing is critical. Putting a motion sensor on a target player makes it possible to automatically extract motion information. However, since it is impossible to ask the opposing team to wear sensors, visual tracking is expected to be a promising way to acquire positional information 5)-9) .In this paper, we focus on badminton. For badminton analysis, we need to estimate and record such information as player position and shot type in each frame 10) . By statistically analyzing the data, important information is provided for improving training and strategy meetings. However, the captured video contains such typical technical issues for visual tracking as small observation size (i.e., low resolution), large variation of player appearance, and partial occlusion. To easily acquire video data for on-site analysis, such data are usually captured by a monocular camera that is fixed at a relatively high place (such as a balcony) to observe all of the court's players (Fig. 1 (upper-left)). The camera needs a wide-angle lens or the distance between the camera and the target players should be long (in Fig. 1, the distance is about 25 m). As shown in Fig. 1 (bottom), the observed size of each player becomes small. Abstract Motion analysis of athletes often provides important information to improve training and strategy meetings. Visual player-tracking techniques are being developed that do not need devices. In this paper, we focus on racket sports, since they suffer from technical issues for visual tracking such as small observation size (low resolution) and large variation of player appearances. Moreover, racket sports video is usually captured by a monocular camera at a set position so that each player is observed at a top and a bottom region of the video across a net on the court. As a result, tracking accuracy is damaged by the net that often occludes players on the far side. As a solution, this paper proposes a method to improve the player-tracking accuracy in badminton video by applying an image pixel compensation technique, such as Image Inpainting. We confirm the effectiveness of our method using videos of badminton singles games.

show abstract

Automatic annotation of tennis games: An integration of audio, vision, and learning

Cited by 26 publications

References 14 publications

On-Device Deep Personalization for Robust Activity Data Collection

On-Device Deep Personalization for Robust Activity Data Collection

Semi-Automated Data Labeling for Activity Recognition in Pervasive Healthcare

[Paper] Improvement of Badminton-Player Tracking Applying Image Pixel Compensation

Contact Info

Product

Resources

About