Background Invariant Faster Motion Modeling for Drone Action Recognition

Abstract

Visual data collected from drones has opened a new direction for surveillance applications and has recently attracted considerable attention among computer vision researchers. Due to the availability and increasing use of the drone for both public and private sectors, it is a critical futuristic technology to solve multiple surveillance problems in remote areas. One of the fundamental challenges in recognizing crowd monitoring videos’ human action is the precise modeling of an individual’s motion feature. Most state-of-the-art methods heavily rely on optical flow for motion modeling and representation, and motion modeling through optical flow is a time-consuming process. This article underlines this issue and provides a novel architecture that eliminates the dependency on optical flow. The proposed architecture uses two sub-modules, FMFM (faster motion feature modeling) and AAR (accurate action recognition), to accurately classify the aerial surveillance action. Another critical issue in aerial surveillance is a deficiency of the dataset. Out of few datasets proposed recently, most of them have multiple humans performing different actions in the same scene, such as a crowd monitoring video, and hence not suitable for directly applying to the training of action recognition models. Given this, we have proposed a novel dataset captured from top view aerial surveillance that has a good variety in terms of actors, daytime, and environment. The proposed architecture has shown the capability to be applied in different terrain as it removes the background before using the action recognition model. The proposed architecture is validated through the experiment with varying investigation levels and achieves a remarkable performance of 0.90 validation accuracy in aerial action recognition.

A Multi-Lingual Speech Recognition-Based Framework to Human-Drone Interaction

“…Another component that might be included to explain the predictions produced by algorithms, as stated in [39], is the explainability of the settings. Moreover, the system can be extended to handle audiovisual interaction techniques, in which the drone can predict the user's movements using an integrated camera and background invariant detection [40]. As the altitude of drones changes, this last strategy must be supplemented with multiscale object detection [41].…”

Skeletal Keypoint-Based Transformer Model for Human Action Recognition in Aerial Videos

“…They used the game data of GTA and FIFA along with GAN generated aerial data from actual ground data for training, and then the model is tested on real aerial data. Kotecha et al [34] designed a Faster Motion Feature Modeling (FMFM) based system with Accurate Action Recognition (AAR) modeling. Their proposed system used a cascade of CNN-based models for both FMFM and AAR.…”

Application of Convolutional Neural Network (CNN) to Recognize Ship Structures

“…Due to various topographical features or irregular objects such as garbage floating in the sea and changes in the surrounding background by weather changes, this may lead to the deterioration of the performance of ship classification and accurate structure recognition. To resolve this problem, various studies suggest adding preprocessing to improve the performance, such as the application of data argumentation or ensemble algorithms to the dataset for AI learning and the background invariant method to prevent performance degradation due to changes in the surrounding background [ 17 , 18 , 19 ]. In addition, it is considered that we need to collect and learn data reflecting various cases such as weather conditions and irregular-sized objects by continuing to fly drones to improve learning performance.…”