A systematic review of publications assessing reliability and validity of the Behavioral Risk Factor Surveillance System (BRFSS), 2004–2011

Abstract

BackgroundIn recent years response rates on telephone surveys have been declining. Rates for the behavioral risk factor surveillance system (BRFSS) have also declined, prompting the use of new methods of weighting and the inclusion of cell phone sampling frames. A number of scholars and researchers have conducted studies of the reliability and validity of the BRFSS estimates in the context of these changes. As the BRFSS makes changes in its methods of sampling and weighting, a review of reliability and validity studies of the BRFSS is needed.MethodsIn order to assess the reliability and validity of prevalence estimates taken from the BRFSS, scholarship published from 2004–2011 dealing with tests of reliability and validity of BRFSS measures was compiled and presented by topics of health risk behavior. Assessments of the quality of each publication were undertaken using a categorical rubric. Higher rankings were achieved by authors who conducted reliability tests using repeated test/retest measures, or who conducted tests using multiple samples. A similar rubric was used to rank validity assessments. Validity tests which compared the BRFSS to physical measures were ranked higher than those comparing the BRFSS to other self-reported data. Literature which undertook more sophisticated statistical comparisons was also ranked higher.ResultsOverall findings indicated that BRFSS prevalence rates were comparable to other national surveys which rely on self-reports, although specific differences are noted for some categories of response. BRFSS prevalence rates were less similar to surveys which utilize physical measures in addition to self-reported data. There is very little research on reliability and validity for some health topics, but a great deal of information supporting the validity of the BRFSS data for others.ConclusionsLimitations of the examination of the BRFSS were due to question differences among surveys used as comparisons, as well as mode of data collection differences. As the BRFSS moves to incorporating cell phone data and changing weighting methods, a review of reliability and validity research indicated that past BRFSS landline only data were reliable and valid as measured against other surveys. New analyses and comparisons of BRFSS data which include the new methodologies and cell phone data will be needed to ascertain the impact of these changes on estimates in the future.

Inter- and independent effects of region and race/ethnicity on variety of fruit and vegetable consumption in the USA: 2011 Behavioral Risk Factor Surveillance System (BRFSS)

“…Finally, all published studies validating the FFQ module for BRFSS were completed for versions prior to 2011, although a similar version of the 2011 FFQ was pilot tested in several states and reliability tested in one state (K Grimm, Centers for Disease Control and Prevention, personal communication, 3 April 2013). Further research is needed to assess the validity and reliability of these and other BRFSS modules as a result of the 2011 changes (54) .…”

A pharmacist-driven academic detailing program to increase adult pneumococcal vaccination

“…6, 18 We evaluated the percentage of adult respondents 65 years and older who have ever had a pneumonia vaccination in Rhode Island and the US. Percent changes in the proportion of respondents answering "yes" to this question over the study period were assessed, as were absolute differences.…”

An adaptation index to high summer heat associated with adverse health impacts in deprived neighborhoods

“…Measurement of the health impacts during high summer heat was developed as a proxy variable for the perceived overall state of health in a heat context. The validity of self-reported versus medical-based diagnoses and behaviours has been well established over time, several countries and data collection methods, especially as a tool for predicting future risks and as an epidemiologic survey tool for prevention and public health actions (Fahimi et al 2008;Jamrozik et al 2014;Pierannunzi et al 2013;Starr et al 1999). The risk group consisted here of respondents saying that they felt (moderately or greatly) the harmful health effects of heat when it was very hot and very humid during the summer (vs. slightly or not at all).…”