Researcher judgment and study design: Challenges of using administrative data

Abstract

Studies linking administrative data from different sources require substantial judgment in constructing research datasets and choosing analytic methods. Moreover, different sample construction rules lead to different results. This suggests that sensitivity analysis should be carried out when alternatives cannot be ruled out. In this case, sensitivity analysis supports the hypothesis of substantial underreporting.

Occupational injury and work organization among immigrant Latino residential construction workers

“…There is general agreement that published injury rates undercount actual injury experience [Azaroff et al, 2002; Boden and Ozonoff, 2010; Schoenfisch et al, 2010], including injury counts in construction [Dong et al, 2011]. Several factors contribute to injury undercounting, including the incomplete scope in the coverage of workers, incomplete capture of reported injury cases, and unreported cases [Ruser, 2010] which can result from employer and employee disincentives to report an injury [Leigh et al, 2004].…”

Advances and Current Themes in Occupational Health and Environmental Public Health Surveillance

“…The approach of using multiple datasets to obtain more complete casecapture has been studied and effectively used in various instances (13,48,106). Capturerecapture has been successfully used to obtain more complete estimates of workplace injuries and illnesses and to assess the undercount of individual surveys (86,105).…”

“…The primary U.S. data source for worker illnesses and injuries is the Department of Labor's (DoL) annual Survey of Occupational Injuries and Illnesses (SOII), which collects data provided by responding employers. The shortcomings of this data source for the purposes of prevention and decision-making have been well documented since the 1980s (13,35,46,55,70,71,85,86,105,117), but little has been done to change the fundamental problems of using an employer-based approach to capture workers' injuries and illnesses in the United States. The Occupational Safety and Health Act of 1970 explicitly excluded government workers (including firefighters and police officers), the self-employed, and many other workers from being included in the DoL's annual survey; this means ∼1 in 5 workers were intentionally excluded.…”

Law, behavior, and research design: A final note

“…We interpret their argument to mean that we [Boden and Ozonoff, 2010]-and by implication others-should not rely on untested assumptions nor should we use sensitivity analyses ''to buttress results.'' Let us examine one assumption made in their response: that BLS reporting rates are the same for different types of cases ''since reporting rules are the same for all cases.''…”