This article presents REVAMP 2 T, Real-time Edge Video Analytics for Multi-camera Privacy-aware Pedestrian Tracking, as an integrated end-to-end IoT system for privacybuilt-in decentralized situational awareness. REVAMP 2 T presents novel algorithmic and system constructs to push deep learning and video analytics next to IoT devices (i.e. video cameras). On the algorithm side, REVAMP 2 T proposes a unified integrated computer vision pipeline for detection, re-identification, and tracking across multiple cameras without the need for storing the streaming data. At the same time, it avoids facial recognition, and tracks and re-identifies pedestrians based on their key features at runtime. On the IoT system side, REVAMP 2 T provides infrastructure to maximize hardware utilization on the edge, orchestrates global communications, and provides system-wide re-identification, without the use of personally identifiable information, for a distributed IoT network. For the results and evaluation, this article also proposes a new metric, Accuracy • Efficiency (AE), for holistic evaluation of IoT systems for real-time video analytics based on accuracy, performance, and power efficiency. REVAMP 2 T outperforms current state-of-the-art by as much as thirteen-fold AE improvement.
With the continuous increase in interstate highway traffic and demand for higher safety standards, there is a growing need for rapidly scalable road inspection. Currently, inspection and condition assessment of roadways involve manual operations which increase labor costs and limit the scalability and inspection coverage. Furthermore, manually inspecting highways adds additional safety risks for highway workers and road inspectors. To address these challenges, we envision a fully automated process of highway inspection. This paper presents a novel low-power drone-mountable real-time artificial intelligence (AI) framework for road asset classification through visual sensing, which is the first step toward a fully automated inspection system. We analyzed a state DOT dataset, consisting of 14 different kinds of defected road assets. To this end, we developed our baseline framework using MobileNet-V2, which is a convolutional neural network (CNN) specially developed for mobile and embedded platforms. Since our target dataset was small and CNNs networks require a huge amount of data, we leveraged transfer learning, by pretraining MobileNet-V2 using the ImageNet dataset and then fine-tuned it on our target dataset. This new framework was ported to embedded platforms Nvidia Jetson Nano with the capability to perform on-board drone processing. Overall, our results demonstrate 81.33% accuracy on the test set while processing 7.4 frames per second and occupying a total power of 1.9 W. It achieved a Power Reduction Factor (PRF) of 21.17 over Nvidia TitanV implementation, with only 8.74% impact on the projected drone flight time.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.