VStore

Abstract

We present VStore, a data store for supporting fast, resourceefficient analytics over large archival videos. VStore manages video ingestion, storage, retrieval, and consumption. It controls video formats along the video data path. It is challenged by i) the huge combinatorial space of video format knobs; ii) the complex impacts of these knobs and their high profiling cost; iii) optimizing for multiple resource types. It explores an idea called backward derivation of configuration: in the opposite direction along the video data path, VStore passes the video quantity and quality expected by analytics backward to retrieval, to storage, and to ingestion. In this process, VStore derives an optimal set of video formats, optimizing for different resources in a progressive manner.VStore automatically derives large, complex configurations consisting of more than one hundred knobs over tens of video formats. In response to queries, VStore selects video formats catering to the executed operators and the target accuracy. It streams video data from disks through decoder to operators. It runs queries as fast as 362× of video realtime.

Multimodal Event Processing: A Neural-Symbolic Paradigm for the Internet of Multimedia Things

“…Modern systems for querying images and video content (i.e., AWStream [25]) rely on extraction techniques based on deep convolutional neural networks (CNNs) due to their accuracy in common computer vision tasks such as classification and object detection (see Figure 3). Visual data is collected and persisted in stable storage in such systems, then analysed either by domain-specific learning models or accelerated by lightweight filters following a store and query processing model (i.e., BlazeIT [26], DeepLens [27], NoScope [28], Optasia [29], Sprocket [30], Tahoma [31], VideoChef [32], and VStore [33]). The most recent and competitive object detection models (Faster R-CNN [34], SSD, YOLOv3 [35], and RetinaNet) have proven to be suitable for image recognition in achieving highperformance results.…”

Towards causal physical error discovery in video analytics systems

“…To reduce the cost of running heavy neural networks on videos, model-level optimizations are developed to make predictions faster while preserving accuracy [13,15]. Other work [8,34] focuses on the storage and decoding of video data, which can be a bottleneck for video analytics as well. [18] envisions a new query system to address new challenges posed by autonomous vehicles (AV) data.…”

“…Correlating abrupt changes in the influence pattern to lighting (e.g., as opposed to object occlusion) can be very challenging if not impossible. There are also scenarios where an engineer might only have access to prediction/explanation logs but not the actual source video, e.g., for privacy reasons [33]. In these cases, an engineer might understand which pixels contribute to a prediction but not what those pixels might represent.…”