Hardware Middleware for Person Tracking on Embedded Distributed Smart Cameras

Abstract

Tracking individuals is a prominent application in such domains like surveillance or smart environments. This paper provides a development of a multiple camera setup with jointed view that observes moving persons in a site. It focuses on a geometrybased approach to establish correspondence among different views. The expensive computational parts of the tracker are hardware accelerated via a novel system-on-chip (SoC) design. In conjunction with this vision application, a hardware object request broker (ORB) middleware is presented as the underlying communication system. The hardware ORB provides a hardware/software architecture to achieve real-time intercommunication among multiple smart cameras. Via a probing mechanism, a performance analysis is performed to measure network latencies, that is, time traversing the TCP/IP stack, in both software and hardware ORB approaches on the same smart camera platform. The empirical results show that using the proposed hardware ORB as client and server in separate smart camera nodes will considerably reduce the network latency up to 100 times compared to the software ORB.

Efficient network clustering for traffic reduction in embedded smart camera networks

“…An asynchronous event exchange among smart camera nodes based on the CORBA Event Service was devised. Based on the empirical results showed in our previous work [34,37], and using the proposed Hardware ORB as client and server in separate smart camera nodes considerably reduced the network end-to-end delays, leading to a performance up to 100 times better than the Software ORB. Even though the Fig.…”

“…The performance of the communication core is enhanced by a hardware implementation of the Object Remote Broker, the HardORB, that was designed in our previous work [34,37]. The HardORB used an offloaded protocol stack operating in hardware and parallel to the embedded processor.…”

“…A very rigorous process here is how to accurately track packets traversing the Linux kernel. We accomplish this task, as explained in [34,37], by defining a variety of probing points at several positions of the Linux kernel's protocol stack, including the network device driver, IP layer, TCP layer, and Socket layer. Also a unique identification (ID) code is assigned to each probing point position.…”

“…To address the complex computations and real-time requirements in embedded smart camera nodes, we designed in our previous work a high-performance and low-power FPGA-based embedded smart camera platform [3,6]. Communication latency among cameras is reduced with the used of a system-on-chip with hardware acceleration of middleware functions, thus enforcing deterministic behavior among the smart camera nodes [33][34][35]37].…”

“…Understanding the correlation between the camera positioning and pathways in the scene on one hand, and temporal and spatial properties of targets on the other hand is key for elaborating a sound messaging infrastructure that can reduce network traffic and latency of message delivery. In this regard, our proposed communication infrastructure assumes a probabilistic modeling for the fusion of camera observations in order to infer a global interpretation of scenes [9,36,37]. Communication is efficiently handled using a task-oriented node clustering that partitions the network in different groups according to the pathways among cameras, and the appearances and temporal behavior of targets.…”