Assessment of clinical competence is complex and inference based. Trustworthy and defensible assessment processes must have favourable evidence of validity, particularly where decisions are considered high stakes. We aimed to organize, collect and interpret validity evidence for a high stakes simulation based assessment strategy for certifying paramedics, using Kane's validity framework, which some report as challenging to implement. We describe our experience using the framework, identifying challenges, decisions points, interpretations and lessons learned. We considered data related to four inferences (scoring, generalization, extrapolation, implications) occurring during assessment and treated validity as a series of assumptions we must evaluate, resulting in several hypotheses and proposed analyses. We then interpreted our findings across the four inferences, judging if the evidence supported or refuted our proposed uses of the assessment data. Data evaluating "Scoring" included: (a) desirable tool characteristics, with acceptable inter-item correlations (b) strong item-total correlations (c) low error variance for items and raters, and (d) strong inter-rater reliability. Data evaluating "Generalizability" included: (a) a robust sampling strategy capturing the majority of relevant medical directives, skills and national competencies, and good overall and inter-station reliability. Data evaluating "Extrapolation" included: low correlations between assessment scores by dimension and clinical errors in practice. Data evaluating "Implications" included low error rates in practice. Interpreting our findings according to Kane's framework, we suggest the evidence for scoring, generalization and implications supports use of our simulation-based paramedic assessment strategy as a certifying exam; however, the extrapolation evidence was weak, suggesting exam scores did not predict clinical error rates. Our analysis represents a worked example others can follow when using Kane's validity framework to evaluate, and iteratively develop and refine assessment strategies.
The proportion of patients with E2B times less than 90 minutes significantly improved through the implementation of a paramedic-activated STEMI bypass protocol. Further study is required to determine whether these benefits are reproducible in other EMS systems.
BackgroundThe incidence of chronic diseases, including diabetes mellitus (DM), heart failure (HF) and chronic obstructive pulmonary disease (COPD) is on the rise. The existing health care system must evolve to meet the growing needs of patients with these chronic diseases and reduce the strain on both acute care and hospital-based health care resources. Paramedics are an allied health care resource consisting of highly-trained practitioners who are comfortable working independently and in collaboration with other resources in the out-of-hospital setting. Expanding the paramedic’s scope of practice to include community-based care may decrease the utilization of acute care and hospital-based health care resources by patients with chronic disease.Methods/DesignThis will be a pragmatic, randomized controlled trial comparing a community paramedic intervention to standard of care for patients with one of three chronic diseases. The objective of the trial is to determine whether community paramedics conducting regular home visits, including health assessments and evidence-based treatments, in partnership with primary care physicians and other community based resources, will decrease the rate of hospitalization and emergency department use for patients with DM, HF and COPD. The primary outcome measure will be the rate of hospitalization at one year. Secondary outcomes will include measures of health system utilization, overall health status, and cost-effectiveness of the intervention over the same time period. Outcome measures will be assessed using both Poisson regression and negative binomial regression analyses to assess the primary outcome.DiscussionThe results of this study will be used to inform decisions around the implementation of community paramedic programs. If successful in preventing hospitalizations, it has the ability to be scaled up to other regions, both nationally and internationally. The methods described in this paper will serve as a basis for future work related to this study.Trial registrationClinicalTrials.gov: NCT02034045. Date: 9 January 2014.Electronic supplementary materialThe online version of this article (doi:10.1186/1745-6215-15-473) contains supplementary material, which is available to authorized users.
Substantial time savings may occur if EMT-Ds/PCPs bypass non-PCI center EDs with only a small predicted increase (about 7 minutes) in the transport time to the PCI center ED. EMT-P/ACP rendezvous does not appear to substantially increase transport time. Given the relatively low occurrence of clinically important events, our findings suggest that EMT-D/PCP bypass to a PCI center ED may be safe and effective for selected STEMI patients.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.