NAPC: A Neural Algorithm for Automated Passenger Counting in Public Transport on a Privacy-Friendly Dataset

Abstract

Real-time load information in public transport is of high importance for both passengers and service providers. Neural algorithms have shown a high performance on various object counting tasks and play a continually growing methodological role in developing automated passenger counting systems. However, the publication of public-space video footage is often contradicted by legal and ethical considerations to protect the passengers' privacy. This work proposes an end-to-end Long Short-Term Memory network with a problem-adapted cost function that learned to count boarding and alighting passengers on a publicly available, comprehensive dataset of approx. 13,000 manually annotated low-resolution 3D LiDAR video recordings (depth information only) from the doorways of a regional train. These depth recordings do not allow the identification of single individuals. For each door opening phase, the trained models predict the correct passenger count (ranging from 0 to 67) in approx. 96% of boarding and alighting, respectively. Repeated training with different training and validation sets confirms the independence of this result from a specific test set.

Engineering the Neural Automatic Passenger Counter

“…[15], which quality-wise was comparable to non-neuronal methods of other manufacturers. Seidel, Jahn, et al introduced the NAPC [8] to predict the boarding and alighting passengers on the Berlin-APC [16] dataset. It consists of roughly 13.000 videos of door opening phases and their total boarding and alighting passenger counts at the video end.…”

“…The maximum amount of passengers in a class (here: boarding or alighting adults) is 67. However, the dataset contains mostly labels with low counts, and videos with high counts are scarce (for details, see Table I within "Number of Events" in [8]).…”

“…The NAPC architecture consists of an input and output fully connected (FC) layer and an LSTM core in between (for details, see Figure 2 in [8]). Therefore, it does not explicitly use spatial information and interprets each video frame as a flattened 500-dimensional vector.…”

“…The output is a regression of the cumulative counted passengers until the corresponding frame. Seidel, Jahn, et al report a maximum overall accuracy of ~97% dropping to 55.17% with their best model in more difficult situations like "More than 20 boarding passengers" [8]. The global relative bias ∆ (global) and an equivalence test for a 95% confidence interval are used as an additional metric, fitting the passenger counting task.…”

“…Seidel, Jahn, et al introduce and evaluate the Neural Automated Passenger Counter (NAPC), a deep neural network based on a long short-term memory (LSTM) [9] architecture [8]. It is capable of learning automatic passenger counting tabula rasa from videos and total manual counts of e.g.…”