Visual Reasoning on Complex Events in Soccer Videos Using Answer Set Programming

Abstract

In the context of computer vision, most of the traditional action recognition techniques assign a single label to a video after analyzing the whole video. We believe that under- standing of the visual world is not limited to recognizing a specific action class or individual object instances, but also extends to how those objects interact in the scene, which im- plies recognizing events happening in the scene. In this paper we present an approach for identifying complex events in videos, starting from detection of objects and simple events using a state-of-the-art object detector (YOLO). We provide a logic based representation of events by using a realization of the Event calculus that allows us to define complex events in terms of logical rules. Axioms of the calculus are encoded in a logic program under Answer Set semantics in order to reason and formulate queries over the extracted events. The applicability of the framework is demonstrated over the scenario of recognizing different kinds of kick events in soccer videos.

Investigation and Development of Methods and Algorithms for Analyzing Video-Eeg Monitoring of Delayed Cerebral Ischemia After Subarachnoid Haemorrhage

“…Note that only a few neural-symbolic systems in the world are based on professional programming languages/libraries like Python (Mojarad et al, 2020, Tarau, 2021, Li et al, 2022, Winters et al, 2022, Ciatto et al, 2022, Kotlin (Ciatto et al, 2021), PyTorch (Manhaeve et al, 2018, Yang et al, 2020, TensorFlow (Li et al, 2022, Ciatto et al, 2022, and Keras (Mojarad et al, 2020). Most neural-symbolic systems are applied only for solving toy problems and only a few neural-symbolic systems applied modern neural architectures like YOLO (Khan et al, 2019) and VGG16 (Padalkar et al, 2023). In contrast to other neural-symbolic systems, the application of the Actor Prolog language let us conduct experiments with any neural architectures and neural network formats supported by the OpenCV library.…”

Improved Soccer Action Spotting using both Audio and Video Streams

“…Some of the most recent works propose to identify kicks and goals in soccer games by using automatic multicamera-based systems [81]. Another work uses logical rules to define complex events in soccer videos in order to perform visual reasoning on these events [39]. These complex events can be visualized as a succession of different visual observations during the game.…”

Representation and Processing of Instantaneous and Durative Temporal Phenomena

“…For example, languages with a point-based temporal model associate facts to instants of time while languages with an interval-based temporal model associate facts with intervals [11], therefore the representation of durative and instantaneous entities in each case respectively is sometimes impossible or not straightforward. Event Calculus [16] approaches [5,10,15] allow the representation of both instantaneous and durative entities, however they lack of interval relations such as those specified by the Allen's interval algebra [2].…”