Archived Intelligent Transportation System Data Quality: Preliminary Analyses of San Antonio TransGuide Data

Abstract

Described are three data quality attributes that are considered relevant to intelligent transportation system (ITS) data archiving: suspect or erroneous data, missing data, and data accuracy. Preliminary analyses of loop detector data from the TransGuide system in San Antonio were performed to identify the nature and extent of these data quality concerns in typical archived ITS data. The findings of the analyses indicated that missing data were inevitable, accounting for about one in five of all possible data records. Error detection rules were developed to screen for suspect or erroneous data, which accounted for only 1 percent of all possible data records. Baseline testing of TransGuide detector accuracy showed mixed results; one location collected traffic volumes within 5 percent of ground truth, whereas traffic volumes at another location ranged from 12 to 38 percent of ground truth. It was concluded that data quality procedures will be essential for realizing the full potential of archived ITS data.

Reliability assessment for traffic data

“…Payne et al (1976) propose identifying data with unreasonable values of flow, speed, or density, while Chen and May (1987) suggest looking for outlying values of occupancy, as compared to historical norms. Similar approaches are also taken by Turner et al (2000) and Chen et al (2003), in which patently impossible data (such as zero volume but positive occupancy) are flagged as suspicious. The Washington algorithm, developed by Nihan et al (1990), uses the concept of an 'acceptable' region of flow-density observations based on traffic theory or historical observations.…”

“…For this reason, even detectors known to malfunction may remain unrepaired for months or years, and the cumulative effect on data quality can be substantial. In a sample of archived data in San Antonio almost a quarter of data records were identified as either 'missing' or 'suspicious' (Turner et al 2000). While missing data is an obvious problem, detecting 'suspicious' or otherwise unreasonable data is somewhat more subtle.…”

Travel Time Prediction Using Multi-layer Feed Forward Artificial Neural Network

“…For example, TransGuide [8] is a traffic condition report system in San Antonio, USA. This system uses cameras, messages, and fiber optic to gather information about accidents, congestions, and constructions.…”

A comparison of outlier detection algorithms for ITS data

“…This procedure is called data quality control or data cleaning. Turner, Albert, Gajewski, and Eisele (2000) have listed several reasons why quality control procedures are especially critical with ITS data.…”