Examining intra-rater and inter-rater response agreement: A medical chart abstraction study of a community-based asthma care program

To, Teresa; Estrabillo, E; Wang, Chengning; Cicutto, Lisa

doi:10.1186/1471-2288-8-29

Cited by 33 publications

(26 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We tested the performance of the data-collection instrument used for medical-record abstraction on 10 patients excluded for these analyses prior to use in the present study. Then, to confirm the accuracy of data taken from medical records, we randomly selected 20 women included in this sample for repeat medical-record review, which represented 5% of the reviews we performed in this study, as recommended in protocols for chart-abstraction reliability testing (To, Estrabillo, Wang, & Cicutto, 2008). Record reviews showed 95.14% of data were accurate, meeting accuracy standards of > 95% (Mi, Collins, Lerner, Losina, & Katz, 2013).…”

Section: Methodsmentioning

confidence: 99%

Oxytocin Augmentation in Spontaneously Laboring, Nulliparous Women: Multilevel Assessment of Maternal BMI and Oxytocin Dose

Carlson

Corwin

Lowe

2017

Biological Research For Nursing

View full text Add to dashboard Cite

Background Synthetic oxytocin, the primary tool for labor augmentation, is less effective among obese women, leading to more unplanned cesarean deliveries for slow labor progress. It is not known if obese women require higher doses of oxytocin due to maternal, fetal, or labor factors related to maternal obesity. Objectives This study had two main objectives: 1) Examine the influence of maternal body mass index (BMI) on hourly doses of oxytocin from augmentation initiation until vaginal delivery in obese women; and 2) Examine the influence of other maternal, fetal, and labor factors on hourly doses of oxytocin in obese women. Study design Longitudinal study of a cohort (N = 136) of healthy, nulliparous, spontaneously laboring obese women (BMI ≥ 30 kg/m2) who received oxytocin augmentation and achieved vaginal delivery. We performed iterative multilevel analyses to examine the influence of maternal BMI and other factors on hourly oxytocin doses. Results Maternal BMI explained 16.56% (95% CI [13.7-20.04], p < 0.001) of the variance in hourly oxytocin doses received in a multilevel model controlling for influence of maternal, fetal, and labor characteristics. Maternal age, gestational age, status of amniotic membranes at hospital admission, and admission cervical dilation examination were not significant; however, neonatal birthweight and cervical dilation at oxytocin initiation were significant predictors of hourly oxytocin dose in these women (p < 0.001). Conclusions Even when parturition preparation has progressed adequately for spontaneous labor initiation, there still may be some obesity-related blunting of myometrial contractility and response to oxytocin used for augmentation.

show abstract

Section: Methodsmentioning

confidence: 99%

Oxytocin Augmentation in Spontaneously Laboring, Nulliparous Women: Multilevel Assessment of Maternal BMI and Oxytocin Dose

Carlson

Corwin

Lowe

2017

Biological Research For Nursing

View full text Add to dashboard Cite

show abstract

“…Kappa adjusts for chance agreement among two ratings. While kappa has been most commonly used to assess agreement between raters (inter-rater reliability), it can also be used to assess similarity or concordance between ratings within the same raters (Fleiss, 1981; Grootendorst, Feeny, & Furlong, 1997; To, Estrabillo, Wang, & Cicutto, 2008). Kappa values greater than .75 may be taken to represent excellent agreement, values in the .40 to .75 range, fair to good agreement, and values less than .40, poor agreement (Fleiss, 1981).…”

Section: Methodsmentioning

confidence: 99%

Comparing barriers to mental health treatment and substance use disorder treatment among individuals with comorbid major depression and substance use disorders

Mojtabai

Chen

Kaufmann

et al. 2014

Journal of Substance Abuse Treatment

View full text Add to dashboard Cite

Barriers to both mental health and substance use disorder treatments have rarely been examined among individuals with comorbid mental health and substance use disorders. In a sample of 393 adults with 12-month major depressive episodes and substance use disorders, we compared perceived barriers to these two types of treatments. Data were drawn from the 2005-2011 US National Surveys on Drug Use and Health. Overall, the same individuals experienced different barriers to mental health treatment vs. substance use disorder treatment. Concerns about negative views of the community, effects on job, and inconvenience of services were more commonly reported as reasons for not receiving substance use disorder treatment. Not affording the cost of care was the most common barrier to both types of treatments, but more commonly reported as a barrier to mental health treatment. Improved financial access through the Affordable Care Act and parity legislation and integration of mental health and substance use disorder services may help to reduce treatment barriers among individuals with comorbid mental health and substance disorders.

show abstract

“…Similarly, To et al 17 found an overall percent agreement of 93% and an overall κ of 0.81 when they examined IRR between the study chart abstractor and an experienced nonstudy chart abstractor, using 8 fi ctitious medical charts.…”

Section: Discussionmentioning

confidence: 99%

“…Finally, there have not been many IRR studies in primary care research, 5,6,17 to assist in determining the lowest threshold of data quality before repeated collection is required. A common interpretation of κ states that a value between 0.61 and 0.80 constitutes substantial agreement between raters, while in emergency medicine research, the benchmark is 95% agreement.…”

Section: In T Er R At Er R El Ia Bil I T Y In Data Col L Ec T Ionmentioning

confidence: 99%

Methods to Achieve High Interrater Reliability in Data Collection From Primary Care Medical Records

Liddy¹,

Wiens²,

Hogg³

2011

The Annals of Family Medicine

View full text Add to dashboard Cite

PURPOSEWe assessed interrater reliability (IRR) of chart abstractors within a randomized trial of cardiovascular care in primary care. We report our fi ndings, and outline issues and provide recommendations related to determining sample size, frequency of verifi cation, and minimum thresholds for 2 measures of IRR: the κ statistic and percent agreement. METHODSWe designed a data quality monitoring procedure having 4 parts: use of standardized protocols and forms, extensive training, continuous monitoring of IRR, and a quality improvement feedback mechanism. Four abstractors checked a 5% sample of charts at 3 time points for a predefi ned set of indicators of the quality of care. We set our quality threshold for IRR at a κ of 0.75, a percent agreement of 95%, or both.RESULTS Abstractors reabstracted a sample of charts in 16 of 27 primary care practices, checking a total of 132 charts with 38 indicators per chart. The overall κ across all items was 0.91 (95% confi dence interval, 0.90-0.92) and the overall percent agreement was 94.3%, signifying excellent agreement between abstractors. We gave feedback to the abstractors to highlight items that had a κ of less than 0.70 or a percent agreement less than 95%. No practice had to have its charts abstracted again because of poor quality.CONCLUSIONS A 5% sampling of charts for quality control using IRR analysis yielded κ and agreement levels that met or exceeded our quality thresholds. Using 3 time points during the chart audit phase allows for early quality control as well as ongoing quality monitoring. Our results can be used as a guide and benchmark for other medical chart review studies in primary care.

show abstract

Examining intra-rater and inter-rater response agreement: A medical chart abstraction study of a community-based asthma care program

Abstract: Background: To assess the intra-and inter-rater agreement of chart abstractors from multiple sites involved in the evaluation of an Asthma Care Program (ACP).

Cited by 33 publications

References 19 publications

Oxytocin Augmentation in Spontaneously Laboring, Nulliparous Women: Multilevel Assessment of Maternal BMI and Oxytocin Dose

Oxytocin Augmentation in Spontaneously Laboring, Nulliparous Women: Multilevel Assessment of Maternal BMI and Oxytocin Dose

Comparing barriers to mental health treatment and substance use disorder treatment among individuals with comorbid major depression and substance use disorders

Methods to Achieve High Interrater Reliability in Data Collection From Primary Care Medical Records

Contact Info

Product

Resources

About