Graphical integrity issues in open access publications: Detection and patterns of proportional ink violations

Abstract

Academic graphs are essential for communicating complex scientific ideas and results. To ensure that these graphs truthfully reflect underlying data and relationships, visualization researchers have proposed several principles to guide the graph creation process. However, the extent of violations of these principles in academic publications is unknown. In this work, we develop a deep learning-based method to accurately measure violations of the proportional ink principle (AUC = 0.917), which states that the size of shaded areas in graphs should be consistent with their corresponding quantities. We apply our method to analyze a large sample of bar charts contained in 300K figures from open access publications. Our results estimate that 5% of bar charts contain proportional ink violations. Further analysis reveals that these graphical integrity issues are significantly more prevalent in some research fields, such as psychology and computer science, and some regions of the globe. Additionally, we find no temporal and seniority trends in violations. Finally, apart from openly releasing our large annotated dataset and method, we discuss how computational research integrity could be part of peer-review and the publication processes.

Replacing bar graphs of continuous data with more informative graphics: are we making progress?

“…A recently developed automated screening tool identifies bar graphs in which the y-axis does not start at zero [ 23 ]. Our data highlight that it would be best to apply this tool only to bar graphs of counts or proportions.…”

An Overview of Recent Work in Media Forensics: Methods and Threats

“…Technologies that can automatically examine figures in published scientific papers will benefit experts who need to accomplish such an important and difficult task. Zhuang et al analyzed graphical integrity issues in open access publications by verifying if the size of shaded areas in scientific figures were consistent with their corresponding quantities [186]. Moreira et al demonstrated a human-inthe-loop end-to-end scientific publication analysis process.…”

A note on creating inset plots using graph twoway

“…We argue, however, that insets can—and should—be used to support purposive communication. In scientific publications, an inset might be a diagram, image, graph, or plot that nudges the eye onto meaningful details or a supportive side story on top of the main message (Tufte 2006; Wainer 2005; Wang et al 2021; Zhuang, Huang, and Acuna 2021). 1…”

“…We argue, however, that insets can-and should-be used to support purposive communication. In scientific publications, an inset might be a diagram, image, graph, or plot that nudges the eye onto meaningful details or a supportive side story on top of the main message (Tufte 2006;Wainer 2005;Wang et al 2021;Zhuang, Huang, and Acuna 2021). 1 Inset plots, in particular, are most typically used to "zoom in" on data that are densely clustered, though they may also be used to view variable distributions, anomalies (without the effects of visual compression), or additional relevant data.…”