Reporting Guidelines for Experimental Research: A Report from the Experimental Research Section Standards Committee

Abstract

The Standards Committee of the Experimental Research Section of the American Political Science Association has produced reporting guidelines that aim to increase the clarity of experimental research reports. This paper describes the Committee's rationale for the guidelines it developed and includes our Recommended Reporting Standards for Experiments (Laboratory, Field, Survey). It begins with a content analysis of current reporting practices in published experimental research. Although researchers report most important aspects of their experimental designs and data, we find substantial omissions that could undermine the clarity of research practices and the ability of researchers to assess the validity of study conclusions. With the need for reporting guidelines established, the report describes the process the Committee used to develop the guidelines, the feedback received during the comment period, and the rationale for the final version of the guidelines.

Transparency, Reproducibility, and the Credibility of Economics Research

“…The only set of disclosure guideline specific to economics that we are aware of is the Consolidated Health Economic Evaluation Reporting Standards, although these appear to be more widely followed in health than in economics (Husereau et al 2013). In this regard, there has been less movement within economics than in other social sciences, including political science, where a section of the American Political Science Association has developed guidelines for reporting of experimental research (Gerber et al 2014). The American Political Science Association has formed committees 34 See https://www.aeaweb.org/PDF_files/PR/ A E A _ A d o p t s _ E x t e n s i o n s _ t o _ P r i n c i p l e s _ f o r _ Author_Disclosure_01-05-12.pdf.…”

Transparency, Reproducibility, and the Credibility of Economics Research

“…The only set of disclosure guideline specific to economics that we are aware of is the Consolidated Health Economic Evaluation Reporting Standards (CHEERS), although these appear to be more widely followed in health than in economics (Husereau et al 2013). In this regard, there has been less movement within economics than in other social sciences, including political science, where a section of the American Political Science Association has developed guidelines for reporting of experimental research (Gerber et al 2014 In psychology, researchers have created an extension of CONSORT for social and psychological interventions (CONSORT-SPI) (Montgomery et al 2013;Grant et al 2013).…”

Institutionalizing Transparency

“…Working groups allow researchers who share an area or method to develop standards that best fit its particularities, as opposed to researchers feeling like standards developed with a different type of research in mind are being imposed on them. For example, in political science, the Experimental Research section of the American Political Science Association convened a subcommittee that offered standards for reporting of different types of experiments (Gerber et al 2014). Such reporting standards make it clear to both authors and reviewers what needs to be included in an article.…”