Pre-validation methods for developing a patient reported outcome instrument

Abstract

BackgroundMeasures that reflect patients' assessment of their health are of increasing importance as outcome measures in randomised controlled trials. The methodological approach used in the pre-validation development of new instruments (item generation, item reduction and question formatting) should be robust and transparent. The totality of the content of existing PRO instruments for a specific condition provides a valuable resource (pool of items) that can be utilised to develop new instruments. Such 'top down' approaches are common, but the explicit pre-validation methods are often poorly reported. This paper presents a systematic and generalisable 5-step pre-validation PRO instrument methodology.MethodsThe method is illustrated using the example of the Aberdeen Glaucoma Questionnaire (AGQ). The five steps are: 1) Generation of a pool of items; 2) Item de-duplication (three phases); 3) Item reduction (two phases); 4) Assessment of the remaining items' content coverage against a pre-existing theoretical framework appropriate to the objectives of the instrument and the target population (e.g. ICF); and 5) qualitative exploration of the target populations' views of the new instrument and the items it contains.ResultsThe AGQ 'item pool' contained 725 items. Three de-duplication phases resulted in reduction of 91, 225 and 48 items respectively. The item reduction phases discarded 70 items and 208 items respectively. The draft AGQ contained 83 items with good content coverage. The qualitative exploration ('think aloud' study) resulted in removal of a further 15 items and refinement to the wording of others. The resultant draft AGQ contained 68 items.ConclusionsThis study presents a novel methodology for developing a PRO instrument, based on three sources: literature reporting what is important to patient; theoretically coherent framework; and patients' experience of completing the instrument. By systematically accounting for all items dropped after the item generation phase, our method ensures that the AGQ is developed in a transparent, replicable manner and is fit for validation. We recommend this method to enhance the likelihood that new PRO instruments will be appropriate to the research context in which they are used, acceptable to research participants and likely to generate valid data.

Linking NHS data for pediatric pharmacovigilance: Results of a Delphi survey

“…Due to the large number of questions asked in Round 1 and in order to cover all relevant TDF domains a systematic data reduction exercise was applied to keep the length of the following rounds manageable. 36 Emphasis was on retaining as much of the original wording and context as possible and care was taken to include minority and majority statements in Round 2. 38 However as in all such Delphi exercises item reduction may result in the loss of some subtleties and nuances contained in Round 1.…”

“…Answers to the free-text questions in Round 1 were paraphrased into individual statements by the lead author (YH). Item reduction followed the method described by Prior et al 36 using item de-duplication, item reduction and removal of content overlap. The final list of items were presented in Round 2 and participants were asked to rate their agreement with these statements on a 9-point, Likert scale (as shown in Fig.…”

Moving from PROMs to POEMs for Glaucoma Care: A Qualitative Scoping Exercise

“…37,38 Prior et al outlined an alternative approach where items in existing instruments are pooled to create instruments with improved content validity. 26 A third solution may be to combine one or more instruments and a final option would be the development of a bespoke instrument, such as the POEM outlined in our study.…”

“…The health outcome subcategories were compared to a database of items derived from 20 existing instruments available in the public domain developed in a previous study. 26 The EQ5D can attract a licensing fee, but was included as it has been adopted in the NHS outcomes framework and by NICE. 27 The howRu instrument was included as it was designed specifically for use in routine practice, and despite its brevity has shown good correlation with established generic health status measures, 28 for a total of 22 instruments.…”

Validation of a free fall acrobatics intervention protocol to reduce neck loads during parachute opening shock

“…Conceptually, our sought measure may be perceived as a form of validity, implicating use of expert rating for assessment. 21 An iterative dialogue was desired, including expert opinion on the relevance and feasibility of the intended intervention protocol.…”