Occupational asthma: an assessment of diagnostic agreement between physicians

Abstract

Objectives: To investigate the levels of agreement between expert respiratory physicians when making a diagnosis of occupational asthma. Methods: 19 cases of possible occupational asthma were identified as part of a larger national observational cohort. A case summary for each case was then circulated to 12 physicians, asking for a percentage likelihood, from the supplied information, that this case represented occupational asthma. The resulting probabilities were then compared between physicians using Spearman's rank correlation and Cohen's k coefficients.Results: Agreement between the 12 physicians for all 19 cases was generally good as assessed by Spearman's rank correlation. For all 66 physician-physician interactions, 45 were found to correlate significantly at the 5% level. The agreement assessed by k analysis was more variable, with a median k value of 0.26, (range -0.2 to +0.76), although 7 of the physicians agreed significantly (p,0.05) with >5 of their colleagues. Only in one case did the responses for probability of occupational asthma all exceed the ''on balance'' 50% threshold, although 12 of the 19 cases had an interquartile range of probabilities not including 50%, implying ''on balance'' agreement. The median probability values for each physician (all assessing the identical 19 cases) varied from 20% to 70%. Factors associated with a high probability rating were the presence of a positive serial peak expiratory flow Occupation Asthma SYStem (OASYS)-2 chart, and both the presence of bronchial hyper-reactivity and significant change in reactivity between periods of work and rest. Conclusions: Despite the importance of the diagnosis of occupational asthma and reasonable physician agreement, certain variations in diagnostic assessment were seen between UK expert centres when assessing paper cases of possible occupational asthma. Although this may in part reflect the absence of a normal clinical consultation, a more unified national approach to these patients is required.

Diagnosis of occupational asthma from time point differences in serial PEF measurements

“…This aids expert interpretation. However, as not all experts agree on the diagnosis of occupational asthma from PEF plots,14 15 reproducible methods of analysis are required. The original Oasys score, based on a discriminant analysis of work-rest-work complexes or their counterpart rest-work-rest complexes, has a sensitivity of 82% and a specificity of 94% for 4-week records with at least four readings per day 5.…”

Agreement in diagnosing occupational asthma by occupational and respiratory physicians who report to surveillance schemes for work-related ill-health

“…A small number of these physicians were ‘experts’ (ie, consultant respiratory physicians with a clinical and research interest in occupational lung disease) who had been involved in a previous study of diagnostic agreement 4. These were excluded.…”

“…The 19 cases used here had been assessed previously (for content) in the study of ‘expert’ respiratory physicians' diagnostic agreement 4. These cases had been selected for that study at random from a large series of patients who had been referred to six secondary care respiratory centres.…”

“…The UK's Revitalising Health and Safety target aimed to reduce the incidence of work-related ill-health by 20% between 1999/2000 and 2009/20102 and guidelines to improve prevention, identification and management of occupational asthma were produced in 2005 3. However, a recent study of diagnostic agreement involving 12 physicians (including nine consultant respiratory physicians with clinical and research interest in occupational lung disease) found wide variation in the probabilities of occupational asthma (SDs ranging from 12% to 25%) and in all cases except one, the range of probabilities crossed 50% 4. Another reliability study investigated the diagnosis and management of occupational asthma among 100 non-specialist respiratory physicians in the UK by asking about participants' diagnostic approaches to a single case scenario, and concluded there was a lack of standardisation and a shortfall of evidence based practice 5…”

Editorial comment

“…In general, the greater the pretest probability, and the greater the number of positive test results, the more confident the physician can be in the diagnosis. There can remain variability, however, even among specialists, in the estimated probability of the diagnosis, as was demonstrated in a recent review of sample cases with test results in the UK [7]. Newer diagnostic tests, such as examination of cells from induced sputum samples [8], may offer the possibility of further enhancing diagnostic confidence, when such tests are available, as reviewed by Dr Lemiere.…”