Standardized Sampling for Systematic Literature Reviews (STAMP Method): Ensuring Reproducibility and Replicability

Abstract

Systematic literature reviews (SLRs) are an effective way of mapping a research field and synthesizing research evidence. However, especially in communication research, SLRs often include diverse theories and methods, which come with a considerable downside in terms of reproducibility and replicability. As a response to this problem, the present article introduces the method of standardized sampling for systematic literature reviews (STAMP). The method is a structured, four-stage approach that is centered around score-based screening decisions. Originating from principles of standardized content analysis, a method common in communication research, and supplementing established guidelines like Cochrane or PRISMA, the STAMP method contributes to more transparent, reproducible, and replicable SLR sampling processes. As we illustrate throughout the article, the method is adaptable to various SLR types. The article also discusses the method’s limitations, such as potential coder effects and comparatively high resource intensity. To facilitate the application of STAMP, we provide a comprehensive guideline via the Open Science Framework that offers a succinct overview for quick reference and includes practical examples for different types of SLRs.

Systematizing destigmatization in the context of media and communication: a systematic literature review

“…A systematic literature review (SLR) is a suitable method for addressing the intended research objectives, as it enables the exploration of a broad, heterogeneous, and interdisciplinary research field in a reproducible and reliable way (Rogge et al, 2024). The methodological approach is based on the recommendations of Petticrew and Roberts (2006) and the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist, as well as corresponding recommendations for systematic literature reviews (Page et al, 2021), and the STAMP method (Rogge et al, 2024). Two types of literature reviews-a scoping review and a theoretical review-were combined to connect the summarization of prior knowledge with further explanation building through a comprehensive search strategy (Paré et al, 2015).…”

“…Neither Krippendorff 's alpha nor Cohen's kappa coefficients were optimal but they reached acceptable values after the second reliability test. Here, it has to be borne in mind that the homogeneity (e.g., a majority of the publications did not fulfill ABS criterion 2) and dichotomy (0/1) of most of the data may have been a problem for the usage of these coefficients (Rogge et al, 2024). During the reliability tests, it was observed that ABS criterion 2-the connection of destigmatization with media or communication-is hard to decide on, as media and communication are broad and diverse concepts.…”