Activity recognition using Video Event Segmentation with Text (VEST)

Holloway, Hillary A.; Jones, Eric K.; Kaluzniacki, Andrew; Blasch, Erik; Tierno, Jorge

doi:10.1117/12.2050413

Cited by 5 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although POL/AD is comprehensive, the approach is unfortunately too expensive to be implemented on fog nodes which host edge units' data streams. Labeling partial video segments rather than bounding boxes in video frames, anomaly analysis segments the video where an action such as moving, stealing, or incident [17], [42]. Video range labeling using a refined Recurrent Neural Network (RNN) also translates to a more accurate rare instance detection along with outputs of the bounding box around the anomalous object [26].…”

Section: B Safety Modeling and Anomaly Detectionmentioning

confidence: 99%

I-SAFE: Instant Suspicious Activity identiFication at the Edge using Fuzzy Decision Making

Nikouei¹,

Chen²,

Aved³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Urban imagery usually serves as forensic analysis and by design is available for incident mitigation. As more imagery collected, it is harder to narrow down to certain frames among thousands of video clips to a specific incident. A real-time, proactive surveillance system is desirable, which could instantly detect dubious personnel, identify suspicious activities, or raise momentous alerts. The recent proliferation of the edge computing paradigm allows more data-intensive tasks to be accomplished by smart edge devices with lightweight but powerful algorithms. This paper presents a forensic surveillance strategy by introducing an Instant Suspicious Activity identiFication at the Edge (I-SAFE) using fuzzy decision making. A fuzzy control system is proposed to mimic the decision-making process of a security officer. Decisions are made based on video features extracted by a lightweight Deep Machine Learning (DML) model. Based on the requirements from the first-line law enforcement officers, several features are selected and fuzzified to cope with the state of uncertainty that exists in the officers decision-making process. Using features in the edge hierarchy minimizes the communication delay such that instant alerting is achieved. Additionally, leveraging the Microservices architecture, the I-SAFE scheme possesses good scalability given the increasing complexities at the network edge. Implemented as an edgebased application and tested using exemplary and various labeled dataset surveillance videos, the I-SAFE scheme raises alerts by identifying the suspicious activity in an average of 0.002 seconds. Compared to four other state-of-the-art methods over two other data sets, the experimental study verified the superiority of the I-SAFE decentralized method.

show abstract

Section: B Safety Modeling and Anomaly Detectionmentioning

confidence: 99%

I-SAFE: Instant Suspicious Activity identiFication at the Edge using Fuzzy Decision Making

Nikouei¹,

Chen²,

Aved³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Individual video clips are exploited for multiple mover tracking, player classification, and team relationships. The text information helps in the segmenting of important activities as surrounded by events boundaries (Holloway, et al, 2014). Note that the same stimulus could be used by a coach trying to instruct lessons learned to his/her players (offense and defense) as well as game highlights.…”

Section: Example 1: Quest Narrativementioning

confidence: 99%

QuEST for Information Fusion in Multimedia Reports

Blasch

Rogers

Holloway

et al. 2014

International Journal of Monitoring and Surveillance Technologies Research

Self Cite

View full text Add to dashboard Cite

Qualia-based Exploitation of Sensing Technology (QuEST) is an approach to create a cognitive exoskeleton to improve human-machine decision quality. In this paper, the authors present QuEST-motivated man-machine information fusion with an example for multimedia narratives. User-based situation awareness includes both elements of external sensory perception and internal cognitive explanation. The authors outline QuEST elements and tenets towards a reasoning approach that achieves human intelligence amplification (IA) in relation to data aggregation from machine artificial intelligence (AI). In a use case example for multimedia exploitation, they showcase the need for enhanced understanding of the man (mind-body cognition) and the machine (sensor-based reasoning) for establishing a cohesive narrative of situational activities. QuEST tenets of structurally coherent, situated conceptualization, and simulated experience are utilized in organizing multimedia reports of Video Event Segmentation by Text (VEST).

show abstract

“…The user is looking at video that includes target tracking, space-time correlation, and clustering. Graphical fusion aids in text-to-video association for event and activity based intelligence (ABI) detection [30]. ABI [31] can support a User-Defined Operating Picture (UDOP) but requires visualization of graphical information fusion results that link the text-and tracking-derived objects graphs.…”

Section: Introductionmentioning

confidence: 99%