Inter‐ and intra‐rater reliability and agreement in determining subcutaneous tumour margins in dogs

Abstract

The objective of this prospective study was to evaluate agreement and reliability of calliper-based measurements of locally invasive subcutaneous malignant tumours in dogs. Four raters measured the longest diameter of 12 subcutaneous tumours (7 soft tissue sarcomas and 5 mast cell tumours) from 11 client-owned dogs during 3 randomized, blinded measurement trials, both pre- and post-sedation. Inter- and intra-rater reliability was evaluated using intra-class correlation coefficient (ICC) and agreement was evaluated using Bland-Altman plots. Inter- and intra-rater reliability was good (ICC range of 0.8694-0.89520) and excellent (ICC range of 0.9720-0.9966), respectively. For agreement calculations, an a priori clinically relevant limit of agreement of 10 mm was set. Inter- and intra-rater agreement was unacceptable with inter-rater limits of agreement ranging from 15.9 to 55.6 mm and intra-rater limit of agreement ranging from 11.9 to 28.1 mm. Review of the measurement trial photographs revealed that calliper orientation changes were frequent, occurring in 9/12 (75%) and 8/12 (67%) pre- and post-sedation cases. No significant correlation was found between inter-rater measurement standard deviations and calliper orientation changes or dog body condition score. These findings suggest veterinarians may have poor agreement in determining the gross edge of tumours, which is expected to introduce bias and inconsistency in tumour staging, assessing response to therapy, and surgical margin planning. Due to the potential consequences for veterinary cancer patients, future studies are needed to validate the present findings.

Intersurgeon agreement in determining 3 cm surgical margins of subcutaneous tumors in dogs

“…The clinically unacceptable variation among surgeons identified in this study is comparable to other peer reviewed veterinary literature assessing caliper-based measurements of lymph nodes and subcutaneous tumors among veterinarians in dogs. 9,10 The findings of the present study integrate well with those previous studies, due to the requirement of identification of the gross edge of the tumor to measure a planned surgical margin length. The prior studies showed that identification of the gross tumor edge varied among different veterinarians, so it is unsurprising that surgeons appear to be delivering varying resection doses even when they are theoretically using the same margin size.…”

“…[2][3][4][5][6][7][8] In previous studies, insufficient agreement has been evident among veterinarians in determining the gross edge of both subcutaneous tumors and lymph nodes in dogs. 9,10 In these studies, veterinarians are consistent within their measurements (ie, intrarater agreement and reliability were good). On the other hand, measurements varied substantially among the raters (ie, interrater agreement was unacceptable).…”

“…On the other hand, measurements varied substantially among the raters (ie, interrater agreement was unacceptable). 9,10 In neither of these studies were the intersurgeon differences in proposed surgical margins directly measured or assessed, yet they raise concerns that surgeons may be delivering different surgical doses despite using a theoretically uniform surgical margin. Such intersurgeon surgical dose variations may at least partly explain the contradictory reports in the peer-reviewed literature regarding the prognostic value of surgical margin size and status.…”

A systematic review of surgical margins utilized for removal of cutaneous mast cell tumors in dogs

“…Most canine MCTs are not biopsied prior to excision due to risk of degranulation so histologic grade is often unknown at the time of surgery. Commonly the extent of gross disease is determined by manual palpation, which has very poor inter-observer agreement, and rarely by advanced imaging [14]. Microscopic mast cell disease can extend beyond palpable gross tumor margins, so traditionally, wide margins of 3 cm laterally and one fascial plane deep have been recommended for resection of all MCTs [15,16].…”

A systematic review of surgical margins utilized for removal of cutaneous mast cell tumors in dogs

“…observer agreement, and rarely by advanced imaging. 14 Microscopic mast cell disease can extend beyond palpable gross tumor margins, so traditionally, wide margins of 3 cm laterally and one fascial plane deep have been recommended for resection of all MCTs. 15,16 Given that resection of these wide margins can cause significant patient morbidity, several investigators have assessed decreased lateral margins and the effect on completeness of excision or local tumor recurrence.…”