Objective To study the inter-observer variation related to extraction of continuous and numerical rating scale data from trial reports for use in meta-analyses. Design Observer agreement study. Data sources A random sample of 10 Cochrane reviews that presented a result as a standardised mean difference (SMD), the protocols for the reviews and the trial reports (n=45) were retrieved. Data extraction Five experienced methodologists and five PhD students independently extracted data from the trial reports for calculation of the first SMD result in each review. The observers did not have access to the reviews but to the protocols, where the relevant outcome was highlighted. The agreement was analysed at both trial and meta-analysis level, pairing the observers in all possible ways (45 pairs, yielding 2025 pairs of trials and 450 pairs of meta-analyses). Agreement was defined as SMDs that differed less than 0.1 in their point estimates or confidence intervals. Results The agreement was 53% at trial level and 31% at meta-analysis level. Including all pairs, the median disagreement was SMD=0.22 (interquartile range 0.07-0.61). The experts agreed somewhat more than the PhD students at trial level (61% v 46%), but not at meta-analysis level. Important reasons for disagreement were differences in selection of time points, scales, control groups, and type of calculations; whether to include a trial in the metaanalysis; and data extraction errors made by the observers. In 14 out of the 100 SMDs calculated at the meta-analysis level, individual observers reached different conclusions than the originally published review. Conclusions Disagreements were common and often larger than the effect of commonly used treatments. Metaanalyses using SMDs are prone to observer variation and should be interpreted with caution. The reliability of metaanalyses might be improved by having more detailed review protocols, more than one observer, and statistical expertise. INTRODUCTIONSystematic reviews of clinical trials, with meta-analyses if possible, are regarded as the most reliable resource
The aim of this study was to measure the validity of the prescriber information recorded in the Danish National Prescription Registry (DNPR). The prescriber information recorded in the pharmacies' electronic dispensing system was considered to represent the prescriber information recorded in the DNPR. Further, the problem of validity of the prescriber information pertains only to non-electronic prescriptions, as these are manually entered into the dispensing system. The recorded prescriber information was thus validated against information from a total of 2000 non-electronic prescriptions at five Danish community pharmacies. The validity of the recorded prescriber information was measured at the level of the individual prescriber and the prescriber type, respectively. The proportion of non-electronic prescriptions with incorrect registrations was 22.4% (95% confidence interval (CI): 20.6-24.3) when considering individual prescriber identifiers and 17.8% (95% CI: 16.1-19.5) when considering prescriber type. When excluding prescriptions specifically registered as 'missing prescriber identifier', the proportions decreased to 9.5% (95% CI: 8.2-11.0) and 4.1% (95% CI: 3.2-5.1), respectively. The positive predictive values for the classification of prescriber types were in the range of 94.0-99.2%, while the sensitivity ranged between 64.6% and 91.8%. With a maximum of 14% non-electronic prescriptions of all prescriptions in the DNPR in 2015, this corresponds to correct classification of prescriber types in the DNPR of at least 97.5%. In conclusion, the prescriber information in the DNPR was found to be valid, especially in recent years. Researchers should be aware of the low sensitivity towards prescriptions from private practicing specialists.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.