Validation of the paediatric appropriateness evaluation protocol in British practice

Abstract

The reliability and validity of the North American paediatric appropriateness evaluation protocol (PAEP) for use in paediatric practice in Britain was tested. The protocol was applied to 418 case records of consecutive emergency admissions to three Yorkshire district general hospitals. The PAEP ratings were then compared with a clinical consensus opinion obtained from two expert panels. Altogether 32% of the admissions were rated inappropriate by the PAEP and 36% by the panels. Validity of the PAEP, as measured by agreement beyond chance with the expert panel rating, was only moderate with a of 0.29 (95% confidence interval 0.11 to 0.47). The PAEP has limited validity for evaluating British paediatric practice. Utilisation review instruments developed in diVering clinical cultures should be used with caution until shown to be valid for the practice setting under review. (Arch Dis Child 1997;77:294-298) Keywords: paediatric appropriateness evaluation protocol; hospitalisation; paediatric admissions Evaluation of appropriateness of an acute hospital admission with measurement tools based on standardised criteria is a relatively new concern in the UK. In North America, such utilisation review instruments have been developed and used since the 1970s, aiming to limit costs and increase eYciency of acute care. The instruments use so called explicit, objective criteria developed for this purpose. One of the widely applied tools is the appropriateness evaluation protocol for use in adult practice (AEP).1 The AEP criteria are based on the level of service provided and, to some extent, factors which identify severity of illness, independent of diagnoses. The protocols were designed for retrospective or concurrent application to case notes. Fulfilment of one of the relevant criteria would make that day of hospital care appropriate. The original criteria in the AEP were tailored to adult specialties. Two versions for children, both called the paediatric appropriateness evaluation protocol (PAEP) were later derived from the adult tool.2 3 The PAEP has been used in several countries, although the methods of application and sampling strategies have been inconsistent making results diYcult to compare. 4 Utilisation reviews are not designed as arbiters of individual patient care. However, results of reviews can influence patient care, service provision, and health policy. If they are unreliable or invalid their use may adversely aVect the quality of health care and unfairly penalise patients; for example admission may be discouraged though needed, or inpatient facilities erroneously reduced.5 On the other hand, evaluation of appropriateness of hospitalisation is a valuable method for monitoring services and informing planning or reconfiguration of services. It is essential, therefore, that before an audit tool is used, reliability and validity of the instruments are suYciently high. Reliability is a measure of reproducibility. Interobserver reliability measures the extent to which raters independently arrive at the sa...

Hospital based alternatives to acute paediatric admission: a systematic review

“…37 There is little consensus about the meaning of terms used or about the collection of routine hospital activity data on acute assessment episodes. For example, the meaning of the term ''admission'' varies between countries; 38 in the studies in this review, paediatric assessment units were typically ''admitting'' patients for two to four hours whereas the A&E units were more likely to hold children for up to 24 hours. Even within the UK, there is still a need for clearly defined activity measures, and the potential pitfalls of misinterpreting routine paediatric inpatient statistics have been highlighted.…”

Reduction of Inappropriate Hospital Admissions of Children With Influenza-Like Illness Through the Implementation of Specific Guidelines: A Case-Controlled Study

“…We used the CPG indications to evaluate the appropriateness of management of children with ILI in terms of admission after accessing the ED. Our working strategy differed from previous studies in 2 aspects: (1) unlike the widely applied general protocols (eg, Pediatric Appropriateness Evaluation Protocol), [6][7][8] the CPG indications for hospital admission were tailored for pediatric ILI and are based on recent data; and (2) the data records came from a direct, case-controlled, prospective observation by specifically trained independent observers rather than from a retrospective record consultation. Using this innovative strategy, we observed a rate of inappropriate admissions Ͼ30%.…”

“…4,5 Most studies of inappropriate admissions have been based on application of the Pediatric Appropriateness Evaluation Protocol, a nonspecific tool used to assess compliance with standard criteria for hospital admission and stay. 4,6,7 High rates of inappropriate admissions for pediatric diseases in general have been reported in various countries. In Australia and Canada, 24% and 29% of admissions, respectively, were considered inappropriate.…”

Unscheduled return visits within 72 hours to an assessment unit

“…In an attempt to address this issue objectively, the paedia tric appropriateness evaluation protocol has been developed,6but proved to have limited validity for evaluating British paediatric practice and so is not recommended 7. Other tools are being evaluated.…”