Generic Metrics for the Estimation of the Prediction Accuracy of Aircraft to Aircraft Conflicts by a Strategic Conflict Probe Tool

Air Traffic Control Quarterly

Bayraktutar

McDonald³

et al. 2010

Unprecedented global initiatives have begun to redesign the aviation systems that provide for the efficient and safe transport of civilian aircraft. Success of these initiatives is only possible through global collaborations that allow broader analyses and data to be shared. The paper reports on just such a study that examines the lateral deviations from the automation's known horizontal route of flight to the actual aircraft position. These errors are due to the typical navigation and surveillance errors, as well as the larger atypical errors that are mainly caused by purposeful changes in the route of flight that are not updated. Large data analyses within the ground automation systems of the United States and Europe indicated errors from 20 to 30 nautical miles are common, while airborne Australian and more samples in the United States had errors from 100 to 800 times smaller. Further analysis illustrated the direct impact these errors have on safety critical separation management functions. It was concluded that airborne derived data via Automatic Dependent Surveillance Contract reports offer a major opportunity to improve the groundbased automation functions.

Section: ) Impact Of Lateral Error On Conflict Predictionsmentioning

confidence: 99%

“…A unitless ratio called the min-max ratio was calculated for all non-conflict encounter events and matched to the associated FA events. The min-max ratio is defined in detail in [18]. To summarize, the maximum ratio between horizontal separation and the horizontal separation standard (e.g.…”

Section: ) Impact Of Lateral Error On Conflict Predictionsmentioning

confidence: 99%

Lateral Intent Error’s Impact on Aircraft Prediction

Air Traffic Control Quarterly

Bayraktutar

McDonald³

et al. 2010

“…Generic metrics for quantitative evaluation are available in Ref. 9. Conflict probe performance metrics are presented in Ref.…”

Section: A Backgroundmentioning

confidence: 99%

“…The categories include (1) the measurements of estimating the error events of the missed and false alerts, (2) the measurements quantifying the timeliness of the correct predictions, valid alerts, (3) the metrics related to the prediction stability of these predictions, and finally (4) prediction sensitivity measurements that quantify the overall performance of the accuracy predictions such as sharpness as defined in Ref. 9. This paper will present definitions of the missed and false alert rates and timeliness metrics.…”

Section: A Backgroundmentioning

confidence: 99%

Performance Metrics for Tactical Aircraft to Aircraft Conflicts

Ryan

AIAA Guidance, Navigation and Control Conference and Exhibit

Hughes³

et al. 2007

An air traffic control system's main function is to separate aircraft. The computer supporting the system assists the air traffic controllers by generating a conflict alert whenever it predicts that two aircraft are about to get too close to each other. The performance of the conflict alert function is a key element to the overall functioning of the air traffic control system. A set of metrics has been designed to measure the conflict alerting performance of an aircraft traffic control system. The key factors are the missed alert rate and the false alert rate. However there are several secondary factors that are essential to measuring the performance, especially in a simulation environment. This paper describes a set of metrics that have been developed to evaluate the performance of the FAA's en route aircraft traffic control system. They have been applied to the existing system, the Host Computer System, and will be used to establish metrics for the new system now under development, the En Route Automation Modernization system. The metrics are calculated by post processing recorded data that has been produced by running a real time simulation of the air traffic system without controllers, using as input field recorded aircraft data that has been time shifted to induce aircraft-to-aircraft conflicts.

Methodology for evaluating and regression testing a conflict probe

The 23rd Digital Avionics Systems Conference (IEEE Cat. No.04CH37576)

Oaks²,

Ryan³

A conflict probe is an air traffic management decision support tool that predicts aircraft-toaircraft and aircraft-to-airspace conflicts. In order to achieve the confidence of the air traffic controllers who are provided this tool, a conflict probe must accurately predict these events. To ensure their continued confidence, the accuracy should not only be assessed in the laboratory before the probe is deployed but continue to be reassessed as the system undergoes upgrades and software changes. Furthermore, it is desirable to use recorded air traffic data to test these tools in order to preserve real-world errors that affect their performance. This paper utilizes a proven approach that modifies surveillance radar track data in time to create traffic scenarios containing conflicts with characteristic properties similar to those encountered in actual air traffic operations. It is these time shifted traffic scenarios that are used to evaluate the conflict probe.evaluating the missed and false conflict predictions, the calculation of the corresponding error probabilities, and a regression testing methodology to examine two runs of the conflict probe to determine if the conflict prediction accuracy has improved or degraded over time. A detailed flight example is presented which illustrates the specific processing involved in conflict accuracy analysis. Next using a scenario of many flights, a methodology utilizing categorical data analysis techniques is applied to determine if a new version of the conflict probe's software significantly improved or degraded in conflict prediction accuracy. This paper describes the detailed process of