Tracking Passengers and Baggage Items Using Multiple Overhead Cameras at Security Checkpoints

Abstract

We introduce a novel framework to track multiple objects in overhead camera videos for airport checkpoint security scenarios where targets correspond to passengers and their baggage items. We propose a Self-Supervised Learning (SSL) technique to provide the model information about instance segmentation uncertainty from overhead images. Our SSL approach improves object detection by employing a test-time data augmentation and a regression-based, rotation-invariant pseudolabel refinement technique. Our pseudo-label generation method provides multiple geometrically-transformed images as inputs to a Convolutional Neural Network (CNN), regresses the augmented detections generated by the network to reduce localization errors, and then clusters them using the mean-shift algorithm. The selfsupervised detector model is used in a single-camera tracking algorithm to generate temporal identifiers for the targets. Our method also incorporates a multi-view trajectory association mechanism to maintain consistent temporal identifiers as passengers travel across camera views. An evaluation of detection, tracking, and association performances on videos obtained from multiple overhead cameras in a realistic airport checkpoint environment demonstrates the effectiveness of the proposed approach. Our results show that self-supervision improves object detection accuracy by up to 42% without increasing the inference time of the model. Our multi-camera association method achieves up to 89% multi-object tracking accuracy with an average computation time of less than 15 ms.

Rare And Popular Event-Based Co-Located Pattern Recognition in Surveillance Videos Using Max-Min PPI-DBSCAN And GREVNN

“…However, the utilized KF cannot recognize objects from videos containing non-Gaussian noise, which restricts the performance. Siddique [23] explored an object detection approach for camera videos using a data augmentation method and CNN. The multiple detections of CNN were clustered by the Mean-Shift Algorithm (MSA).…”

“…. , 𝑆 𝑞 } (23) Where, (𝑞) is the number of features extracted. Next, from (𝑆), optimal features are selected as mentioned below.…”

“…Thus, the respective person co-located with the object was not recognized. • For the effective tracking of passengers and their baggage in [23] the presence of complex background, noise, occlusions, illumination variations, and ghost effects was not concentrated. • Confusion and tracking loss occur between the passenger and their co-locating objects.…”

Irregular luggage classification experiments using data from Tromsø airport

“…Indeed, examples of image recognition in the context of classification and tracking of luggage are shown in [9][10][11][12][13][14][15][16]. Convolutional neural networks (CNNs) were used in [9,10] to identify and classify abandoned items.…”

“…Convolutional neural networks (CNNs) were used in [9,10] to identify and classify abandoned items. In [11], a multi-camera setup with transfer learning was used to track passengers and their luggage at security checkpoints in an airport. Perhaps the most well-documented application of object recognition and image processing in relation to airports and luggage is the detection of illegal items in x-ray images from baggage scanners [12][13][14][15].…”