|
|
email Dr. SETI ®  |
|
Near Midair Collisions: How Many Really Occur?
Copyright © 1993 by H. Paul Shuch, Ph.D.
|
|
email Dr. SETI ® |
|
Short Title: Near Midair Collision Sample Rate
Abstract
Aircraft midair collision (MAC) accidents, though thankfully rare events, are a major safety concern which has motivated the development of elaborate air traffic control (ATC) systems. Yet ATC system effectiveness is difficult to assess because MACs occur so infrequently as to defy valid statistical analysis. An alternative measure, near midair collisions (NMACs), is often used to gauge system effectiveness. However, significant selective reporting biases make the NMAC database a weak indicator of actual collision trends. The rate of NMAC reporting may well be more highly correlated with risk perception than with actual collision risk.
This paper attempts to deduce the actual incidence of NMACs in the United States, through application of three different analysis techniques. The different methods appear to converge weakly toward a solution with an interesting interpretation: although perhaps one flight in a million ends in a midair collision accident, roughly one flight in a thousand passes close enough to another aircraft to constitute a NMAC. Thus the ATC system successfully mitigates on the order of 99.9% of all potential midair collisions.
Key Words
Midair Collision, Near Midair Collision, Air Traffic Control, Aviation Safety Reporting System.
Introduction
The possibility of an in-flight encounter between aircraft has existed ever since Wilbur first said to Orville, "Let's build another one." Fortunately the midair collision (MAC) is a rare event, accounting for less than one percent of all aviation accidents. There are currently about 60 Million flights per year conducted in the United States. For the past two decades, the number of midair collision accidents has averaged about 30 per year. Since each midair collision is presumed to involve two aircraft, it appears that one flight in a million ends in a midair collision. MAC safety studies have been frustrated by the combination of high data variability and small population size.
Near midair collisions (NMACs), which occur far more frequently than collision accidents, provide us with abundant data to support safety analysis. Two assumptions are implicit in NMAC studies: that every MAC began as a near midair (whether recognized or not), and that every reported NMAC somehow failed to escalate into a collision accident (and thus represents a successful collision avoidance episode). Thus we can hope to gain from NMAC reports useful insights, which may help us to prevent MAC accidents.
The criterion most commonly used to define the NMAC incident is that two aircraft pass within 500 feet of each other, or are in such proximity that an aircrew member reports that a collision hazard existed. The FAA has been collecting pilot- initiated NMAC reports since 1959, and generally receives about 1,000 such reports per year [1]. There is reason to suspect that significant selective reporting is taking place. FAA NMAC investigation procedures [2] specify: "Existing radar, communication, and weather data will be examined in the conduct of the investigation. When possible, all cockpit crew members will be interviewed regarding factors involving the NMAC incident . . . When investigation reveals a violation of an FAA regulation, enforcement action will be pursued."
In fact "the vast majority of near midair collision reports fail to indicate a violation or an assignable error on the part of either the operators of the aircraft or air traffic control personnel" [3]. Still, it was felt that limited immunity from enforcement action would encourage greater pilot participation in NMAC reporting.
In 1976 the NASA Aviation Safety Reporting System was established to "stimulate a free flow of information regarding potential deficiencies in the aviation system. . . The ASRS program provides for waiver of certain disciplinary actions, particularly those against pilots and controllers, if reports are submitted in a timely fashion. The program also guarantees anonymity to persons filing reports. Unlike reports submitted directly to the FAA, ASRS reports are not independently investigated and their accuracy is not verified. Therefore, ASRS reports are subject to strong reporting biases" [4]. Pilot initiated ASRS NMAC reports number about 300 per year [5].
Not all NMAC incidents are reported to either the FAA or ASRS, thus the databases represent two presumably independent samples of some larger underlying population. Since the number of reports submitted to the two NMAC databases exhibits little temporal consistency, it is difficult to determine the size of that underlying population. The purpose of this study is to determine whether the NMAC sample size is adequate to support statistically significant safety analysis.
Prior Studies
Between 1986 and 1989, an Interagency Near Midair Collision Working Group met semiannually at FAA headquarters to review the most current NMAC data and the progress of action plans related to the Working Group recommendations [4, 6]. Membership in the Working Group has included representatives of the FAA; NASA; US Air Force, Navy, Army and Coast Guard; Office of the Secretary of Transportation; and the National Transportation Safety Board. Working Group discussions have concentrated on the characteristics of the NMAC population without attempting to estimate its size.
Yesley [7] recognizes that NMAC and MAC data are difficult to compare, because the former represent a sample of incidents, and the latter a population. His study explores the various types of bias to which NMAC reports are subject, including non- respondent bias. He estimates that at least three-fourths of NMAC incidents may go unreported, but I am not clear on how he arrives at this figure. He further states that 75 to 95 percent of air carrier pilots involved in NMACs report them. The implication is that air carrier pilots are little subject to reporting bias. But this estimate was apparently obtained by comparing air carrier NMAC pilot reporting frequency to that of NMACs detected by Air Traffic Control (ATC) radar in the terminal environment. The majority of MACs occur outside of radar coverage, as do a considerable fraction of General Aviation (GA) NMACs.
In an earlier study the FAA Office of Aviation Safety [8] explored NMACs involving a GA aircraft and a military or air carrier flight. It was stated that less than 20 percent of these incidents were reported by the GA operator, although it appears to this investigator that the percentage applied to those flights reported by the air carrier or military operator. The statistic suggests that GA operators may be under-reporting NMACs to a far greater extent than the FAA acknowledges.
In comparing NMAC reporting frequency between the FAA and ASRS databases, the Office of Technology Assessment [9] found that about 18 percent of the air carrier- involved NMAC reports in the FAA database also showed up in ASRS reports. Overall, fewer than ten percent of NMACs reported to the FAA also appear in the ASRS data. OTA states that "air carrier NMAC data are more consistent than the other subsets of the data, including the total." Since the vast majority of MAC accidents involve only GA aircraft (and fortunately, Air Carrier MACs are extremely rare), it appears NMAC reporting frequency may have little bearing on studies of the MAC problem.
The Aviation Safety Commission [10] also compared ASRS to FAA NMAC reporting frequencies, noting a 124% increase in FAA NMAC reports between 1983 and 1987, while ASRS reports declined 17.6% over the same period. The Commission has been unable to determine which might be the more reliable indicator. Their report notes "a change in the number of reported incidents could be due to a change in the underlying risk of midair collision, a change in the propensity to report incidents, or simply a change in the number of aircraft operations."
An Empirical Solution
With an overlap of less than ten percent, we can presume the FAA and ASRS NMAC databases to be independent samples drawn from the same population (of NMAC flights). The question of sampling ratio must now be addressed. We know sampling bias exists in each of the two databases, thus can assume that some number of NMACs goes unreported. How significant is this number? Regarding the visual acquisition of traffic called by ATC, the author's personal experience as a commercial pilot and flight instructor suggests that about 90% of intruder aircraft may go undetected. The Assumption of Mediocrity would imply that other pilots do no better nor worse than the author in this regard. Further, there are a number of reasons why pilots exhibit a natural reticence to formally report NMACs. Of pilots interviewed in the author's capacity as an FAA Accident Prevention Counselor, perhaps one in ten actually filed a written incident report. Thus the NMAC sample rate can be rather crudely estimated at perhaps (0.1) · (0.1), or about one percent.
A More Analytical Approach
Since no obvious way to determine the actual number of annual NMACs presents itself, an admittedly arbitrary technique is proposed for approximating this number (and from it, NMAC sampling rate) from its upper and lower limits. This is not a scientific solution, but still yields interesting results.
After weeding out duplicates, we find that in 1986 the two databases received reports on 1082 separate NMACs. Clearly we can infer from this statistic that in 1986 no fewer than 1082 NMACs occurred, a lower limit. It should be immediately apparent that this figure represents an unrealistically low estimate of NMACs, because it suggests perfect detection and no selective reporting whatever.
At the other extreme, in 1986 the US civil aviation segments (air carrier and GA; military figures not available) logged 56.6 Million flights. If we assume that every single flight resulted in a NMAC, and given that NMACs involve two aircraft, we can set as an upper limit the figure of 28.3 Million NMACs for that year. Clearly, this estimate is no less absurd than the previous one.
We have established that the number of NMACs to occur in 1986 falls somewhere between 1082 and 28.3 Million -- a range of more than four orders of magnitude! We know the true figure is unlikely to be near either of the limits, thus would presume a mathematical expectation related to some sort of central tendency.
The data gives us no compelling reason to consider one estimate within the middle of that range to be any more likely than any other. However, since midair collision accidents (which bear an as-yet-unspecified relation to NMACs) are rare events, it is reasonable to expect the actual NMAC number to fall somewhat nearer to the lower than the upper limit. Because of this, and since the dynamic range of our estimate is great, we propose that a reasonable estimate of the mathematical expectation of annual NMACs may be somewhere near the geometric mean of the two limits. That is,
N actual ~ (N min · N max) ^ (1/2) [Equation 1]
The square root of the product (1082 x 2.83·107) yields on the order of 175,000 NMACs for the year being considered. A guess, to be sure, but no less plausible than any other guess.
The above NMAC estimate, as compared to the 1082 reported NMACs in the year being evaluated, suggests that something like one NMAC in 162 was reported, for a sample rate of about 0.6 percent. This figure correlates well with our rather arbitrary empirical estimate. Of course, the calculation also implies that one flight in 162 passes within close proximity to another aircraft - a figure which pilots will find disquieting, but not altogether unreasonable.
Deriving A Probabilistic Solution
Odoni [11] has suggested a strategy previously employed by the FAA, whereby the overall NMAC population size and sampling rate can be derived from the fraction of NMACs which are reported by both pilots. We assume that the pilots report independently. The fact that one pilot did, or did not, report a NMAC can in no way influence the likelihood that the other pilot will file a report. Of course, the circumstances and severity of the individual incident may well influence both pilots' reporting probability. Nevertheless, we assume here that the aircrews operated independently. Intuitively, the population size should thus be to the sample size as the sample size is to the number of NMACs reported twice.
Some time ago, the author derived from Odoni's suggestion a set of equations for estimating the probability that a NMAC would be reported, as well as an estimate of the total number of NMACs which occur each year (whether reported or not). Since the results differ somewhat from the prior art (and from conventional wisdom), the derivations are included in Appendix A. The equations themselves are shown below.
If we let:
p = X2 / (X1 + X2) [Equation 2]
and n = { (X1 + X2) ^2 } / X2 [Equation 3]
If we express Equation 3 as a proportion, we see that:
X2 : Xt :: Xt : n [Equation 4]
which is precisely what we would expect intuitively.
Applying the Probabilistic Solution
Now to determine the size n of the underlying NMAC population, all that remains is to determine X1 and X2, the number of NMACs reported by one and both aircraft, respectively. As Table I indicates, for the period 1983 to 1989, ASRS received a total of 2680 NMAC reports, including 55 cases in which the incident was reported by both pilots. Applying Equation 3, we find for that period a mean annual NMAC population of 20,318, with a standard deviation of 3,783. Similarly, from Equation 2, the mean NMAC reporting probability is 0.021, with a standard deviation of 0.006.
What do these numbers mean? Statistically, we can now be 95% confident that the actual number of annual NMACs falls somewhere between 12,903 and 27,733, and that the actual percentage of those NMACs reported to the NASA ASRS lies between 0.9% and 3.3%.
Some interesting trends in NMAC reporting frequency are evident in Table I. For example, in 1987 (the year following the catastrophic Cerritos midair accident), the number of NMACs reported to ASRS jumped 46%. However, that same year the number of NMACs reported by both pilots more than doubled. We see that the estimated number of NMACs remains stable following the Cerritos tragedy; only the reporting probability is seen to vary. Although NMAC reporting frequency is clearly tied to levels of awareness, the actual number of NMACs which occur more closely follows flight activity levels.
Conclusions
The empirical, analytical and probabilistic estimates of the population size of near midair collisions differ widely, yet fall within the same order of magnitude, and lead us to the conclusion that NMACs are not such rare events after all. It is fortunate that the majority of these close encounters are of such a nature as to pose little risk of escalating into a midair collision accident.
The probabilistic solution estimates that the ASRS sample captures, on the average, about 2% of all NMACs. At the opposite extreme, the geometric mean computation suggests the two databases together sampled perhaps 0.6% of the population of NMACs. The empirical solution yields an intermediate estimate of perhaps one percent. Our results are summarized in Table II.
Each of our three solutions yields an estimated sample size adequate to infer from the sample the characteristics of the underlying population of NMACs. Statistical analyses in the physical and behavioral sciences typically achieve acceptable results with sample sizes on the order of a fraction of a percent.
Of course, adequacy of sample size is a necessary, but not a sufficient, condition for assuring the validity of the data. Whether the suspected reporting biases in the two NMAC databases will preclude meaningful analysis, only further inspection of the data can determine. We expect that the contents of NMAC reports will prove more interesting than their quantity.
If we can accept any of these results as reasonable, it follows that perhaps one tenth of one percent of NMACs actually escalates into a MAC accident. Since every NMAC, whether reported or not, constitutes a successful collision avoidance episode, we conclude that the ATC system is effectively performing its primary task of collision prevention.
Acknowledgments
This research has been supported by a most generous Fellowship from the Fannie and John Hertz Foundation. Thanks are also due to the staff of the NASA Aviation Safety Reporting System, for supplying the data which went into deriving Table I.
References
TABLE I
NMAC Population Estimated From ASRS Reports
| ASRS | ASRS | Estimated | Estimated | |
|---|---|---|---|---|
| Year | Xt | X2 | p [Eq. 2] | n [Eq. 3] |
| 1983 | 226 | 2 | 0.009 | 25,538 |
| 1984 | 289 | 6 | 0.021 | 13,920 |
| 1985 | 333 | 6 | 0.018 | 18,482 |
| 1986 | 363 | 6 | 0.017 | 21,962 |
| 1987 | 528 | 14 | 0.027 | 19,913 |
| 1988 | 457 | 11 | 0.024 | 18,986 |
| 1989 | 484 | 10 | 0.021 | 23,426 |
| ------ | ------ | ------ | ------ | ------ |
| t | 2680 | 55 | n/a | 142,226 |
| m | 383 | 8 | 0.021 | 20,318 |
| S | 110 | 4 | 0.006 | 3,783 |
TABLE II
Comparison of Various NMAC Estimates
| Method: | Estimated Annual NMACs | Corresponding NMAC Report Sample Rate |
|---|---|---|
| Empirical | 108,000 | 1.0 % [Based upon FAA + ASRS reports] |
| Analytical | 175,000 | 0.6 % [Based upon FAA + ASRS reports] |
| Probabilistic | 20,000 | 2.1 % [Reported to ASRS only] |
Appendix A
Derivation of Equations 2 through 4
Since NMACs involve two aircraft, an underlying assumption of the probabilistic solution is that if a NMAC is reported with probability p, it will be reported by both pilots with probability p^2. If we let:
and since X1 + X2 = Xt
it follows that
X1 + X2 = np
The assumption of independence suggests that
X2 = np^2 [Equation B]
thus we determine that
Cross-multiplying,
p = X2 / (X1 + X2) [Equation 2]
and finally, inserting [2] into [B], we get
X2 = n { X2 / (X1 + X2) }^2
so that n = (X1 + X2)^2 / X2 [Equation 3]
If we express [3] as a proportion, we see that:
X2 : Xt :: Xt : n [Equation 4]
which is precisely what we would expect intuitively.
|
Copyright © H. Paul Shuch, Ph.D.; Maintained by Microcomm this page last updated 14 June 2007 |
 TOP |