The standard way to assess medical technologies is to conduct a randomized clinical trial. Patients are randomly assigned to groups receiving alternative treatments, and outcomes are monitored over a long period of time. For example, some victims of left main coronary artery disease may undergo coronary artery bypass surgery, and others may receive medical treatment with nitroglycerine and beta blockers. Comparison of five-year mortality and morbidity in the two groups helps to determine the relative appropriateness of the two procedures. In addition, information about quality of life and cost can also be collected and compared.

We measured geographic differences in the use of medical and surgical services during 1981 by Medicare beneficiaries (age greater than or equal to 65) in 13 large areas of the United States. The average number of Medicare beneficiaries per site was 340,000. We found large and significant differences in the use of services provided by all medical and surgical specialties. Of 123 procedures studied, 67 showed at least threefold differences between sites with the highest and lowest rates of use. Use rates were not consistently high in one site, but rates for procedures used to diagnose and treat a specific disease varied together, as did alternative treatments for the same condition. These results cannot be explained by the actions of a small number of physicians. We do not know whether physicians in high-use areas performed too many procedures, whether physicians in low-use areas performed too few, or whether neither or both of these explanations are accurate. However, we do know that the differences are too large to ignore and that unless they are understood at a clinical level, uninformed policy decisions that have adverse effects on the health of the elderly may be made.

We convened three panels of physicians to rate the appropriateness of a large number of indications for performing a total of six medical and surgical procedures. The panels followed a modified Delphi process. Panelists separately assigned initial ratings, then met in Santa Monica, California where they received reports showing their initial ratings and the distribution of the other panelists' ratings. They discussed the indications and revised the indications

The authors compare the appropriateness ratings and mutual influence of panelists from different specialties rating a comprehensive set of indications for six surgical procedures. Nine-member panels rated each procedure: abdominal aortic aneurysm surgery, carotid endarterectomy, cataract surgery, coronary angiography, and coronary artery bypass graft surgery/percutaneous transluminal coronary angioplasty (common panel). Panelists individually rated the appropriateness of indications at home and then discussed and re-rated the indications during a 2-day meeting. Subsequently, they rated the necessity of those indications scored by the group as appropriate. There were 45 panelists, including specialists (either performers of the procedure or members of a related specialty) and primary care providers, all drawn from nominations by their respective specialty societies. Main outcome measures included: individual panelists' mean ratings over all indications, mean change and conformity scores between rounds of ratings, and the percentage of audited actual procedures rated appropriate or necessary. Performers had the highest mean ratings, followed by physicians in related specialties, trailed by primary care providers. One fifth of all actual procedures were for indications rated appropriate by performers and less than appropriate by primary care providers. At the panel meetings, primary care providers and related specialists showed no greater tendency to be influenced by other panelists than did performers. Multispecialty panels provide more divergent viewpoints than panels composed entirely of performers. This divergence means that fewer actual procedures are deemed performed for appropriate or necessary indications.