Measuring quality for use in incentive schemes: The case of “shrinkage” estimators

Abstract

Researchers commonly “shrink” raw quality measures based on statistical criteria. This paper studies when and how this transformation's statistical properties would confer economic benefits to a utility‐maximizing decision‐maker across common asymmetric information environments. I develop the results for an application measuring teacher quality. The presence of a systematic relationship between teacher quality and class size could cause the data transformation to do either worse or better than the untransformed data. I use data from Los Angeles to confirm the presence of such a relationship and show that the simpler raw measure would outperform the one most commonly used in teacher incentive schemes.

Assessing the quality of public services: For‐profits, chains, and concentration in the hospital market

“…published metrics and using nonlinear policy cut-offs (e.g., penalty/no-penalty), which may frequently change as the basis for reward or penalization of providers. While our findings are specific to the HRRP, our approach to the extraction of a medium-run measure of quality from an observed binary signal of performance could be applied in similar settings to understand the factors associated with quality (Mehta, 2019). 25 Hospitals in Maryland are exempted from the policy due to a special Medicare agreement.…”

“…When considering hospital behavior, measures of quality that relate to incentivized metrics such as those in HRRP should reflect heterogeneity in strategic decisions (Mehta, 2019). This may differ by for‐profit status.…”

“…24 24 Many attempts to incentivize the provision of public services share with the HRRP a similar structure of being based on annually published metrics and using nonlinear policy cut‐offs (e.g., penalty/no‐penalty), which may frequently change as the basis for reward or penalization of providers. While our findings are specific to the HRRP, our approach to the extraction of a medium‐run measure of quality from an observed binary signal of performance could be applied in similar settings to understand the factors associated with quality (Mehta, 2019). …”

A New Method for Estimating Teacher Value-Added

“…In this context, a Type I error occurs when a teacher is ranked below 5% while her true quality ranking is above 5%; conversely, a Type II error occurs when a teacher is ranked above 5% when her true quality is below 5%.30 For instance,Mehta (2019) finds that policymakers should not use the empirical Bayes correction under certain systematic relationships between teacher quality and class size.…”