Test-Retest Reliability of Web-Based Retrospective Self-Report of Tobacco Exposure and Risk

Abstract

Background Retrospectively collected data about the development and maintenance of behaviors that impact health are a valuable source of information. Establishing the reliability of retrospective measures is a necessary step in determining the utility of that methodology and in studying behaviors in the context of risk and protective factors.Objective The goal of this study was to examine the reliability of self-report of a specific health-affecting behavior, tobacco use, and its associated risk and protective factors as examined with a Web-based questionnaire.Methods Core tobacco use and risk behavior questions in the Lifetime Tobacco Use Questionnaire—a closed, invitation-only, password-controlled, Web-based instrument—were administered at a 2-month test-retest interval to a convenience sample of 1229 respondents aged 18 to 78 years. Tobacco use items, which covered cigarettes, cigars, smokeless tobacco, and pipe tobacco, included frequency of use, amount used, first use, and a pack-years calculation. Risk-related questions included family history of tobacco use, secondhand smoke exposure, alcohol use, and religiosity.Results Analyses of test-retest reliability indicated modest (.30 to .49), moderate (.50 to .69), or high (.70 to 1.00) reliability across nearly all questions, with minimal reliability differences in analyses by sex, age, and income grouping. Most measures of tobacco use history showed moderate to high reliability, particularly for age of first use, age of first weekly and first daily smoking, and age at first or only quit attempt. Some measures of family tobacco use history, secondhand smoke exposure, alcohol use, and religiosity also had high test-retest reliability. Reliability was modest for subjective response to first use.Conclusions The findings reflect the stability of retrospective recall of tobacco use and risk factor self-report responses in a Web-questionnaire context. Questions that are designed and tested with psychometric scrutiny can yield reliable results in a Web setting.

Development and Reliability of the Lifetime Interview on Smoking Trajectories

“…The pattern of reliability documented here, highly consistent with findings based on the web-based LTUQ (Brigham et al, 2008(Brigham et al, , 2009(Brigham et al, , 2010, further delineates the relative salience of different smoking history variables to smokers and former smokers. For example, the age one progresses to daily smoking appears to be a highly salient benchmark, while smoking rate and minutes to first cigarette during that initial phase are less consistently recalled.…”

“…To reduce selection bias in the reliability sample, monetary incentives for the retest interview were high relative to the baseline interview; the 64% of the sample who participated did not differ from those who had been invited but did not participate. Our retest rate compares favorably to the retest rates below 35% obtained by Brigham et al (2008Brigham et al ( , 2009) in their web-based studies, which offered smaller financial incentives. Future test-retest research may benefit from the use of higher value incentives to facilitate recruitment and retention at retest.…”

“…In large samples, the ICC has an F distribution, which was used to derive asymptotic 95% CI estimates. Following Brigham et al (2008Brigham et al ( , 2009, indices of reliability ranging from .70 to 1.0 were considered high, .50-.69 were considered moderate, .30-.49 were considered modest, and <.30 were considered low. Other interpretive guidelines exist, however; for example, Landis and Koch (1977) rated k as .80-1.0 = almost perfect, .60-.80 = substantial, .40-.60 = moderate, .20-.40 = fair, .00-.20 = slight, and <.00 = poor.…”

“…The most pertinent research addressing these issues has been conducted by Brigham et al (Brigham et al, 2008, 2010, developers of the Lifetime Tobacco Use Questionnaire (LTUQ), a web-based retrospective assessment. Their first two studies evaluated test-retest reliability of self-administered web-based LTUQ variables across 2-year and 2-month intervals in adult samples of convenience, respectively (Brigham et al, 2008(Brigham et al, , 2009). In the third study, smoking data collected in two separate prospective cohort studies were used to test the validity of retrospective LTUQ data collected 3.9 years later in one sample and 19.5 years later in the other (Brigham et al, 2010).…”

Consumer perceptions of direct-to-consumer personalized genomic risk assessments

“…Finally, the SGHI relies on web-based, self-report assessment of health behaviors. Although there is ample precedent for taking this approach 34 and it allowed us to enroll a large number of individuals, self-report, in particular, is known to be less reliable than in-person clinical assessment.…”