What has been learned about electronic health data as a primary data source for regulatory decisions regarding the harms of drugs? Observational studies with electronic health data for postmarket risk assessment can now be conducted in Europe and the US in patient populations numbering in the tens of millions compared with a few hundred patients in a typical clinical trial. With standard protocols, results can be obtained in a few months; however, extensive research published by scores of investigators has illuminated the many obstacles that prevent obtaining robust, reproducible results that are reliable enough to be a primary source for drug safety decisions involving the health and safety of millions of patients. The most widely used terminology for coding patient interactions with medical providers for payment has proved illsuited to identifying the adverse effects of drugs. Directly conflicting results were reported in otherwise similar patient health databases, even using identical event definitions and research methods. Evaluation of some accepted statistical methods revealed systematic bias, while others appeared to be unreliable. When electronic health data studies detected no drug risk, there were no robust and accepted standards to judge whether the drug was unlikely to cause the adverse effect or whether the study was incapable of detecting it. Substantial investment and careful thinking is needed to improve the reliability of risk assessments based on electronic health data, and current limitations need to be fully understood.
Key PointsElectronic health data for postmarket surveillance became a key element in the new paradigm for drug regulation, which involved fewer and smaller clinical trials prior to marketing approval.The research programs and pilot systems created to study harms of licensed drugs proved largely unable to provide credible evidence of new, unsuspected drug adverse effects, and conflicting and contradictory results when seeking to confirm known harms.Major problems included a limited underlying terminology, few validation studies, and the need for additional statistical standards for these complex data.