Detecting Errors and Imputing Missing Data for Single-Loop Surveillance Systems

Abstract

Single-loop detectors provide the most abundant source of traffic data in California, but loop data samples are often missing or invalid. A method is described that detects bad data samples and imputes missing or bad samples to form a complete grid of clean data, in real time. The diagnostics algorithm and the imputation algorithm that implement this method are operational on 14,871 loops in six districts of the California Department of Transportation. The diagnostics algorithm detects bad (malfunctioning) single-loop detectors from their volume and occupancy measurements. Its novelty is its use of time series of many samples, instead of basing decisions on single samples, as in previous approaches. The imputation algorithm models the relationship between neighboring loops as linear and uses linear regression to estimate the value of missing or bad samples. This gives a better estimate than previous methods because it uses historical data to learn how pairs of neighboring loops behave. Detection of bad loops and imputation of loop data are important because they allow algorithms that use loop data to perform analysis without requiring them to compensate for missing or incorrect data samples.

Active traffic management on road networks: a macroscopic approach

“…If, on the other hand, detector data are poor owing to a malfunctioning detector or just because the capacity is never reached at this point of the freeway (figure 4b), then we can either use fundamental diagrams from the neighbouring links or impute the missing data as suggested in Chen et al (2003) and repeat steps 2-4. Figure 4 illustrates the fact that measurements on the free-flow side of the fundamental diagram are usually represented by a straight line, whereas measurements in the congested region tend to be more scattered, which may justify the case for alternative forms of the fundamental diagram (e.g.…”

The Role of Non-Recurring Congestion in Massive Hurricane Evacuation Events

“…An algorithm using the Support Vector Machine (SVM) pattern classifier was proposed by Chen, Kwon, Rice, Skabardonis, and Varaiya (2003). The SVM pattern classifier sorts out an input vector into one of two classes with a decision boundary developed based on the concept of structural risk minimization of classification error, using statistical learning theory.…”

Improved Dual-Loop Detection System for Collecting Real-Time Truck Data

“…Chen, et al [35], developed a diagnostics algorithm to detect malfunctioning singleloop detectors from their volume and occupancy measurements. Unlike previous approaches, the algorithm employed a time series of many samples, rather than basing decisions on a single sample.…”