How to measure non‐consistency of medical practices with available evidence in therapeutics: a methodological framework

Abstract

Since the early 1980s many studies showed a gap between available evidence and medical practice. They were designed to assess the real impact of randomized clinical trials on the practice of medicine. Their results substantiated a knowledge translation problem. However, they were qualitative findings, i.e. a gap exists or not, although the problem is quantitative (how large is the gap?) and has several components that should be documented according to the objective of the study. In this article, we explored the components and the various contexts in which the measure of the distance between practice and knowledge is considered. All these features should be taken into account for a more accurate and relevant assessment of the distance.

Medical Records Documentation of HIV/AIDS Clinical Services at Primary Health Care (PHC) Facilities and Its Implications on Continuum of Care and Operational Research in South Africa

“…There was no continuity of clinical data documentation from the HCT registers to the clinical stationery. Our results also confirmed other findings on non-consistency of data documentation, which led to a less accurate and incomplete assessments [20].…”

“…The present study confirms the findings that record-keeping in these settings was inadequate. It is recommended that there should be continuous training, monitoring and evaluation of health providers regarding record-keeping issues [20]. Supply of adequate recording materials and proper time management amongst nurses to improve record-keeping is essential and requires to be implemented in all health care facilities [23]…”

New tools to measure discrepancy between prescribing practices and guideline recommendations

“…Measuring deviation requires that the following dimensions of therapeutic decision making be taken into account: (i) the disease to treat: in most cases, this is ‘the diagnosis made’ rather than ‘the real illness’; (ii) the treatment prescribed and, sometimes, its characteristics such as the dosage regimen, the duration, the cost, etc. ; (iii) the therapeutic objective – such as increased life expectancy – considering the patient’s characteristics and wishes or the disease severity; (iv) the LOE: this is an attribute of the available evidence regarding the treatment efficacy; LOE may also be considered as a weighting coefficient of deviation; and (v) the patient, his clinical characteristics, concomitant treatments, basic risks of presenting a morbid event or dying, and preferences in terms of quality of life [5].…”

“…Assessing that discrepancy involves three main steps [5]: (i) setting reference prescriptions; (ii) collecting prescribing practices; and (iii) measuring deviation between medical practices and these references. However, many studies carried out since the 1980s to assess the relevance of medical practices in comparison with some sort of knowledge, norms or guidelines on drug prescription exhibited three weaknesses: (i) lack of a precise definition of non‐conformity, discrepancy, or ‘deviation’, between the available guidelines and actual prescriptions; (ii) inappropriate definition of the reference treatments; and, mostly (iii) lack of an adequate and standardized tool to measure that deviation [5]. Indeed, deviation was often represented by a binary indicator (1 for conformity, 0 for discrepancy) while it is multidimensional and can hardly be summarized by a yes/no assessment without loss of information and possibly loss of sensitivity.…”

Evaluation of two evidence‐based knowledge transfer interventions for physicians. A cluster randomized controlled factorial design trial: the CardioDAS Study

“…Twenty simulated cases (five questions in each case) had to be answered by each participant for both sessions 1 and 2. Conformity was coded for each line of prescription as follows: (1) perfect conformity (prescription in accordance with evidence for condition, compound, dose, schedule and therapeutic objective; or absence of prescription when no high level of evidence is available for it) [27]; (2) absence of conformity (prescription without available evidence or erroneous therapeutic objective; absence of prescription in spite of high level of evidence for it); (3) debatable conformity (poorly described condition, assumption of a class effect, low level of evidence, therapeutic objective not clinically relevant or inaccurate); (4) 'obvious' conformity (absence of randomized controlled trial but obvious clinical efficacy, e.g. For the latter, all prescriptions made during 1 week for sessions 1 and 2 were collected.…”