A descriptive study of samples sizes used in agreement studies published in the PubMed repository

Abstract

Introduction A sample size justification is required for all studies and should give the minimum number of subjects to be recruited for the study to achieve its primary objective. The aim of this review is to describe sample sizes from agreement studies with continuous or categorical endpoints and different methods of assessing agreement, and to determine whether sample size justification was provided. Methods Data were gathered from the PubMed repository with a time interval of 28th September 2018 to 28th September 2020. The search returned 5257 studies of which 82 studies were eligible for final assessment after duplicates and ineligible studies were excluded. Results We observed a wide range of sample sizes. Forty-six studies (56%) used a continuous outcome measure, 28 (34%) used categorical and eight (10%) used both. Median sample sizes were 50 (IQR 25 to 100) for continuous endpoints and 119 (IQR 50 to 271) for categorical endpoints. Bland–Altman limits of agreement (median sample size 65; IQR 35 to 124) were the most common method of statistical analysis for continuous variables and Kappa coefficients for categorical variables (median sample size 71; IQR 50 to 233). Of the 82 studies assessed, only 27 (33%) gave justification for their sample size. Conclusions Despite the importance of a sample size justification, we found that two-thirds of agreement studies did not provide one. We recommend that all agreement studies provide rationale for their sample size even if they do not include a formal sample size calculation.

A community-based intervention study involving family gardens with aromatic herbs on changes in dietary and urinary sodium

“…Bland Altman plot was built to see the concordance between both variables (dietary and urine Na), we assessed graphically the assumptions verifying no discernible shape in the graph [ 19 ]. Previous publications with small sample sizes (between 15 and 30) used this same method to evaluate concordance [ 20 ]. Linear regression models were run in order to estimate the contribution of each NOVA group Na from salt and from seasoning powder to total dietary Na, using the post-estimation Shapley value decomposition, in order to assess the relative contributions of each predictor variable (percentage of contribution to R 2 ) on the total Na intake.…”

Concordance between electronic health record‐recorded race/ethnicity and parental report in hospitalized children

“…We used χ 2 and Fisher's exact tests, as appropriate, to compare proportions between demographic groups. Because of some uncertainty over how best to calculate a sample size for an agreement study, 13 especially in this case where we were evaluating the potential for insufficient rather than sufficient agreement, we opted to power this study based on the survey question “how comfortable do you feel about your child's race and ethnicity being displayed in our electronic medical record.” We hypothesized that parents belonging to historically marginalized demographic groups would be more uncomfortable with the display of race/ethnicity in the EHR. To calculate our sample size to detect a difference between Hispanic and non‐Hispanic populations, we anticipated (based on pilot surveys) that ~20% of respondents would be uncomfortable with race/ethnicity display.…”

Personalized pain management: The relationship between clinical relevance and reliability of measurements

“…The last parameter to consider is the sample size of the reliability study (Han et al, 2022). In this regard, there are numerous recommendations for sample size estimation (Gerke et al, 2022), based for example on the expected (Walter et al, 1998) and related precision or power considerations (Shieh, 2014a, 2014b), the expected width of the (Jan & Shieh, 2018; Lu et al, 2016), or combinations (Mokkink et al, 2022).…”