Time Shifting Air Traffic Data for Quantitative Evaluation of a Conflict Probe

Paglione, Mike; Hughes, W. J.; Oaks, Robert D.; Summerill, J. S.; Township, Egg Harbor

doi:10.2514/6.2003-5343

Cited by 6 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In normal operations very few actual conflicts occur -not enough for testing. It was necessary to time shift the data to create conflicts for the system to predict 10,11 . The conflicts in the scenario are pseudo-conflicts as they did not actually occur except in the simulation.…”

Section: B Simulationmentioning

confidence: 99%

Performance Metrics for Tactical Aircraft to Aircraft Conflicts

Ryan

Paglione

Hughes³

et al. 2007

AIAA Guidance, Navigation and Control Conference and Exhibit

Self Cite

View full text Add to dashboard Cite

An air traffic control system's main function is to separate aircraft. The computer supporting the system assists the air traffic controllers by generating a conflict alert whenever it predicts that two aircraft are about to get too close to each other. The performance of the conflict alert function is a key element to the overall functioning of the air traffic control system. A set of metrics has been designed to measure the conflict alerting performance of an aircraft traffic control system. The key factors are the missed alert rate and the false alert rate. However there are several secondary factors that are essential to measuring the performance, especially in a simulation environment. This paper describes a set of metrics that have been developed to evaluate the performance of the FAA's en route aircraft traffic control system. They have been applied to the existing system, the Host Computer System, and will be used to establish metrics for the new system now under development, the En Route Automation Modernization system. The metrics are calculated by post processing recorded data that has been produced by running a real time simulation of the air traffic system without controllers, using as input field recorded aircraft data that has been time shifted to induce aircraft-to-aircraft conflicts.

show abstract

Section: B Simulationmentioning

confidence: 99%

Performance Metrics for Tactical Aircraft to Aircraft Conflicts

Ryan

Paglione

Hughes³

et al. 2007

AIAA Guidance, Navigation and Control Conference and Exhibit

Self Cite

View full text Add to dashboard Cite

show abstract

“…[5][6][7][8][9] Regression Testing that the new release of the conflict probe software is as accurate as the baseline system. In the context of conflict prediction accuracy, the comparison is to determine if more Missed and False Alerts are present in the new conflict probe relative to the baseline version.…”

Section: Ementioning

confidence: 99%

Methodology for evaluating and regression testing a conflict probe

Paglione

Oaks²,

Ryan³

The 23rd Digital Avionics Systems Conference (IEEE Cat. No.04CH37576)

View full text Add to dashboard Cite

A conflict probe is an air traffic management decision support tool that predicts aircraft-toaircraft and aircraft-to-airspace conflicts. In order to achieve the confidence of the air traffic controllers who are provided this tool, a conflict probe must accurately predict these events. To ensure their continued confidence, the accuracy should not only be assessed in the laboratory before the probe is deployed but continue to be reassessed as the system undergoes upgrades and software changes. Furthermore, it is desirable to use recorded air traffic data to test these tools in order to preserve real-world errors that affect their performance. This paper utilizes a proven approach that modifies surveillance radar track data in time to create traffic scenarios containing conflicts with characteristic properties similar to those encountered in actual air traffic operations. It is these time shifted traffic scenarios that are used to evaluate the conflict probe.evaluating the missed and false conflict predictions, the calculation of the corresponding error probabilities, and a regression testing methodology to examine two runs of the conflict probe to determine if the conflict prediction accuracy has improved or degraded over time. A detailed flight example is presented which illustrates the specific processing involved in conflict accuracy analysis. Next using a scenario of many flights, a methodology utilizing categorical data analysis techniques is applied to determine if a new version of the conflict probe's software significantly improved or degraded in conflict prediction accuracy. This paper describes the detailed process of

show abstract