Establishing an appropriate mode of comparison for measuring the performance of marbling score output from video image analysis beef carcass grading systems1,2

Abstract

A beef carcass instrument grading system that improves accuracy and consistency of marbling score (MS) evaluation would have the potential to advance value-based marketing efforts and reduce disparity in quality grading among USDA graders, shifts, and plants. The objectives of this study were to use output data from the Video Image Analysis-Computer Vision System (VIA-CVS, Research Management Systems Inc., Fort Collins, CO) to develop an appropriate method by which performance of video image analysis MS output could be evaluated for accuracy, precision, and repeatability for purposes of seeking official USDA approval for using an instrument in commerce to augment assessment of quality grade, and to use the developed standards to gain approval for VIA-CVS to assist USDA personnel in assigning official beef carcass MS. An initial MS output algorithm was developed (phase I) for the VIA-CVS before 2 separate preliminary instrument evaluation trials (phases II and III) were conducted. During phases II and III, a 3-member panel of USDA expert graders independently assigned MS to 1,068 and 1,242 stationary carcasses, respectively. Mean expert MS was calculated for each carcass. Additionally, a separate 3-member USDA expert panel developed a consensus MS for each carcass in phase III. In phase II, VIA-CVS stationary triple-placement and triple-trigger instrument repeatability values (n = 262 and 260, respectively), measured as the percentage of total variance explained by carcasses, were 99.9 and 99.8%, respectively. In phases II and III, 95% of carcasses were assigned expert MS for which differences between individual expert MS, and for which the consensus MS in phase III only, was < or = 96 MS units. Two differing approaches to simple regression analysis, as well as a separate method-comparability analysis that accommodates error in both dependent and independent variables, were used to assess accuracy and precision of instrument MS predictions vs. mean expert MS. Method-comparability analysis was more appropriate in assessing the bias and precision of instrument MS predictions. Ether-extractable fat percentages (n = 257; phase II) differed among MS (P < 0.05) but were not suitable to predict or validate assigned MS. The performance and reproducibility of expert MS assignment in future evaluations was considered, and an official USDA performance standard was established, to which an instrument must conform to be approved for official on-line MS assessment. The VIA-CVS subsequently was approved to assign MS to carcasses on-line after completion of a 2006 USDA instrument approval trial conducted according to methods developed during completion of this study.

Prediction of fat percentage within marbling score on beef longissimus muscle using 3 different fat determination methods1

“…Strong, positive correlations (P < 0.0001) were observed between marbling score and fat percentage for each method: CEM (r = 0.91), ether (r = 0.93), and Folch (r = 0.90; data not presented in tabular form). Similar correlations were also observed in previous research (McBee and Wiles, 1967;Armbruster et al, 1983;Moore et al, 2010).…”

“…Fat percentage increased as marbling score increased (P < 0.05) regardless of fat extraction method (Table 4). These results agree with previous research where ether was used and indicated that fat percentage increased with an increase in marbling scores (McBee and Wiles, 1967;Savell et al, 1986;Moore et al, 2010). Strong, positive correlations (P < 0.0001) were observed between marbling score and fat percentage for each method: CEM (r = 0.91), ether (r = 0.93), and Folch (r = 0.90; data not presented in tabular form).…”

“…In this study, least squares means of ether-extractable fat were numerically greater for Slightly Abundant (SLAB), Moderate (MD), and Modest (MT) and less for Small (SM) and Slight (SL) compared with Savell et al (1986). Compared with Moore et al (2010), means for ether-extractable fat in this study were numerically greater for SLAB, MD, and SL and less for MT and SM. Garcia et al (2006) reported the same relative relationships using microwave drying and NMR with the CEM SMART Trac for determining percentage of fat within LM.…”

“…As marbling score increased, variation in fat percentage within each method increased as well (Table 4; Figure 1). Moore et al (2010) also reported an increase in fat percentage variance as marbling score increased and that both expert panel graders and camera grading precision decreased as marbling score increased. Garcia et al (2006) found similar results of variation in fat percentage using the CEM.…”

“…To meet PRIME I standards, the instrument had to demonstrate accuracy and precision of marbling score with 95% repeatability (USDA, 2006a). Studies show that this system was very accurate and over 98% repeatable at commercial production speeds (Woerner and Belk, 2008;Moore et al, 2010). Before instrument assessment of marbling score can be fully implemented, it must meet both PRIME I and Performance Requirements for Instrument Marbling Evaluation II (PRIME II) requirements before approval by the USDA (USDA, 2006a,b).…”