Analysis of common methodological flaws in the highest cited e-cigarette epidemiology research

Abstract

The prevalence of vaping, also known as using e-cigarettes, vapes and vape pens, has prompted a demand for reliable, evidence-based research. However, published literature on the topic of vaping often raises concerns, characterized by serious flaws and a failure to adhere to accepted scientific methodologies. In this narrative review, we analyze popular vaping studies published in medical journals that purport to evaluate the association of vaping and smoking cessation, smoking initiation or health outcomes. We analyzed 24 included studies to identify the questions they claimed to address, stated methods, manner of implementation, discussions, and stated conclusions. After critical appraisal, we noted a multiplicity of flaws in these studies, and identified patterns as to the nature of such flaws. Many studies lacked a clear hypothesis statement: to the extent that a hypothesis could be inferred, the methods were not tailored to address the question of interest. Moreover, main outcome measures were poorly identified, and data analysis was further complicated by failure to control for confounding factors. The body of literature on “gateway” theory for the initiation of smoking was particularly unreliable. Overall, the results and discussion contained numerous unreliable assertions due to poor methods, including data collection that lacked relevance, and assertions that were unfounded. Many researchers claimed to find a causal association while not supporting such findings with meaningful data: the discussions and conclusions of such studies were, therefore, misleading. Herein, we identify the common flaws in the study design, methodology, and implementation found in published vaping studies. We present our summary recommendations for future vaping research. Our aim is to prompt future researchers to adhere to scientific methods to produce more reliable findings and conclusions in the field of vaping research.

A Close Look at Vaping in Adolescents and Young Adults in the United States

“…According to the MTF survey (15), past 30-day EC use was reported by 17.2% of 12th graders, but only 6.6% had used ECs for > 5 days in the past month (24,25). It is important that measures of EC use capture frequency, intensity, reasons for EC use and concomitant cigarette smoking to provide a comprehensive framework for assessing the public health impact of ECs (26)(27)(28). In particular, one-time or experimental use of EC is highly unlikely to increase an individual's risk for adverse health effects.…”

“…Propagation of common mistakes in epidemiological surveys requires urgent critical review and reform. Common flaws in the methodology of epidemiology research have been critically analyzed, including lack of a clear hypothesis statement, methods that were not tailored to address the question of interest, poor characterization of outcome measures, failure to control for relevant confounding factors, and claim of causal association unsupported by the data (28).…”