TIRADS Interobserver Variability Among Indeterminate Thyroid Nodules: A Single‐Institution Study

Abstract

Objectives-A high proportion of cytologically indeterminate, Afirma Gene Expression Classifier "suspicious" thyroid nodules are benign. The Thyroid Imaging Reporting and Data System (TIRADS), was proposed by the American College of Radiology in 2017 to help classify thyroid nodules based on ultrasound characteristics in a standardized fashion to guide management. We aim to determine the interobserver variability of TIRADS classification among cytologically indeterminate and Afirma suspicious nodules.Methods-We retrospectively queried cytopathology archives for thyroid fineneedle aspiration specimens obtained between February 2012 and September 2016 with associated (1) indeterminate diagnosis, (2) ultrasound imaging at our institution, (3) Afirma suspicious result, and (4) surgery at our institution. We compared the TIRADS variability of the 3 blinded radiologists using intraclass correlation coefficients.Results-Our cohort consisted of 127 nodules. Intraclass correlation coefficients can be interpreted as follows: less than 0.4, poor; 0.4 to 0.59, fair; 0.6 to 0.74, good; 0.75 to 1.00, excellent. The intraclass correlation coefficients of the raw TIRADS score and category variability was 0.561 (95% confidence interval [CI]: 0.464-0.651) or fair and 0.547 (95% CI, 0.449-0.640) or fair, respectively. When analyzing composition, echogenicity, shape, margin, and echogenic foci, the ICCs were 0.552 (95% CI, 0.454-0.643), fair; 0.533 (95% CI, 0.432-0.627), fair; 0.359 (95% CI, 0.248-0.469), poor; 0.192 (95% CI, 0.084-0.308), poor; and 0.549 (95% CI, 0.451-0.641), fair, respectively.Conclusions-Our results show that among the subset of cytologically indeterminate and Afirma suspicious nodules, TIRADS interobserver variability was fair. Shape and margin criteria were the biggest sources of disagreement. Large prospective studies are needed to evaluate the interobserver variability of TIRADS in this subset of thyroid nodules.

Reducing the Number of Unnecessary Thyroid Nodule Biopsies With the American College of Radiology (ACR) Thyroid Imaging Reporting and Data System (TI-RADS)

“…Interobserver variability among radiologists and sonographers is an important consideration to improve TI-RADS performance. In a retrospective analysis of 127 nodules using ACR TI-RADS, Sahli et al (2019) showed that while TI-RADS interobserver variability was fair (0.6 to 0.74), shape and margin criteria were the biggest sources of disagreement (poor; 0.359 and 0.192, respectively) [25]. Interestingly, in a separate study exploring sonographer performance and interobserver variability, Wildman-Tobriner et al (2020) showed that sonographers also struggle with margins.…”

2020 Chinese guidelines for ultrasound malignancy risk stratification of thyroid nodules: the C-TIRADS

“…In addition, observer variabilities of TIRADS regarding the interpretation of ultrasound features or FNA recommendation are receiving increasing attention [67][68][69]. Experienced and less experienced radiologists may have different interpretations of thyroid nodule ultrasound features and therefore requires a continuous and specialized training for residents to improve the agreement and performance thyroid ultrasound in residents [70,71].…”

Summary of meta-analyses of studies considering lesion size cut-off thresholds for the assessment of eligibility for FNAB and sonoelastography and inter- and intra-observer agreement in estimating the malignant potential of focal lesions of the thyroid gland

“…A number of studies have been published on the interobserver agreement in the assessment of individual thyroid ultrasound imaging features, which ranges from poor to almost perfect, depending on the feature and study (18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30)(31)(32)(33) . Liu et al (34) conducted a meta-analysis of seven studies assessing inter-observer agreement published up to December 2018, including a total of 927 patients (18)(19)(20)(21)(22)(23) .…”

“…A number of studies have been published on the interobserver agreement in the assessment of individual thyroid ultrasound imaging features, which ranges from poor to almost perfect, depending on the feature and study (18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30)(31)(32)(33) . Liu et al (34) conducted a meta-analysis of seven studies assessing inter-observer agreement published up to December 2018, including a total of 927 patients (18)(19)(20)(21)(22)(23) . They calculated the pooled agreement between examiners in the assessment of individual features in thyroid ultrasound images, with the following results: substantial agreement for structure (0.61; 95% Cl: 0.55-0.66) and presence of calcifications (0.71; 95% Cl: 0.65-0.77), moderate agreement for echogenicity (0.58; 95% Cl: 0.51-0.64), shape (0.53; 95% Cl: 0.45-0.62), and the presence of echogenic foci, including punctate echogenic foci/microcalcifications, macrocalcifications, peripheral calcifications, and comet tail artefacts (0.43; 95% Cl: 0.32-0.54), and fair agreement for margins (0.40; 95% Cl: 0.32-0.48).…”