Multi-sensor fusion for sport genre classification of user generated mobile videos

Multimodal machine learning (MML) is a tempting multidisciplinary research area where heterogeneous data from multiple modalities and machine learning (ML) are combined to solve critical problems. Usually, research works use data from a single modality, such as images, audio, text, and signals. However, real-world issues have become critical now, and handling them using multiple modalities of data instead of a single modality can significantly impact finding solutions. ML algorithms play an essential role by tuning parameters in developing MML models. This paper reviews recent advancements in the challenges of MML, namely: representation, translation, alignment, fusion and co-learning, and presents the gaps and challenges. A systematic literature review (SLR) applied to define the progress and trends on those challenges in the MML domain. In total, 1032 articles were examined in this review to extract features like source, domain, application, modality, etc. This research article will help researchers understand the constant state of MML and navigate the selection of future research directions.

show abstract

“…The sports domain has 4 papers: 2 in classification [387], [388] and 1 in recognition [389] and segmentation [390].…”

Section: Inclusion Criteriamentioning

confidence: 99%

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

2023

View full text Add to dashboard Cite

show abstract

“…Auxiliary sensors embedded in smartphones, such as magnetometers and accelerometers, were already used for the task of video analysis, such as in [7] for sport classification and in [8] for salient event detection in videos of live music shows. However, this paper proposes completely novel algorithms for jointly analyzing visual and magnetometer data in order to discover salient events in sport videos.…”

Section: Prior Artmentioning

confidence: 99%

Salient Event Detection in Basketball Mobile Videos

Cricri

Mate

Curcio

et al. 2014

2014 IEEE International Symposium on Multimedia

Self Cite

View full text Add to dashboard Cite

Modern smartphones have become the most popular means for recording videos. In fact, thanks to their portability, smartphones allow for recording anything and at any moment of our everyday life. One common occasion is represented by sport happenings, where people often record their favourite team or players. Automatic analysis of such videos is important for enabling applications such as automatic organization, browsing and summarization of the content. This paper proposes novel algorithms for the detection of salient events in videos recorded at basketball games. The novel approach consists of jointly analyzing visual data and magnetometer data. The magnetometer data provides information about the horizontal orientation of the camera. The proposed joint analysis allows for a reduced number of false positives and for a reduced computational complexity. The algorithms are tested on data captured during real basketball games. The experimental results clearly show the advantages of the proposed approach.

show abstract

Video key-frame extraction for smart phones

Liu

Sun

2014

Multimed Tools Appl

View full text Add to dashboard Cite

Multi-sensor fusion for sport genre classification of user generated mobile videos

Cited by 7 publications

References 11 publications

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

Salient Event Detection in Basketball Mobile Videos

Video key-frame extraction for smart phones

Contact Info

Product

Resources

About