A Missed Opportunity for Clarity: Problems in the Reporting of Effect Size Estimates in Infant Developmental Science

Abstract

Several years ago, the American Psychological Association began requiring that effect size estimates be reported to provide a better indication of the associative strength between factors and dependent measures in empirical studies (Publication manual of the American Psychological Association, 2010, Author, Washington, DC). Accordingly, developmental journals require/ strongly recommend effect size estimates be included in published work. Potentially, this trend has important benefits for infancy research given some of the inherent difficulties in establishing conceptually strong findings when often facing highly variable performance in typically small samples. This study examined recent infant research from select journals for accuracy and interpretative value of effect size estimates. Demographics, sample size, design, and statistical data were coded from 158 published (2007)(2008)(2009)(2010)(2011)(2012) articles presenting 878 effect size estimates from experimental findings with infants using behavioral methods. Descriptive and distribution statistics were calculated for the following variables: (1) statistical tests, (2) effect size parameters, and (3) effect size interpretations. Although partial eta squared (g p 2 ) and eta squared (g 2 ) were most common (49 and 42%, Correspondence should be sent to Laura Mills-Smith or Derek P

Promoting Replicability in Developmental Research Through Meta‐analyses: Insights From Language Acquisition Research

“…Nevertheless, when possible, it seems important to consider the paradigm being used, and possibly use a more sensitive way of measuring infants’ capabilities. One reason that researchers do not appear to choose the most robust methods might again be due to a lack of consideration of meta‐analytic effect size estimates, which in turn might be (partially) due to a lack of information on (how to interpret) effect size estimates and lack of experience using them for study planning (Mills‐Smith, Spangler, Panneton, & Fritz, ). We, thus, recommend to change this practice and take into account the possibility that different methods’ sensitivity is reflected in effect size.…”

“…A possible reason for prospective power calculations and meta‐analyses being rare lies in the availability of data in published reports. Despite longstanding recommendations to move beyond the persistent focus on p values (such as American Psychological Association, ), a shift toward effect sizes or even the reporting of them has not (yet) been widely adopted (Mills‐Smith et al., ).…”

“…We suggest reporting the following information, in line with current guidelines: means and standard deviations of dependent measures being statistically analyzed (for within‐participant designs with two dependent variables, correlations between the two should be added), test statistic, exact p value (when computed), and effect sizes (e.g., Cohen's d as used in the present article) where possible. Such a standard not only follows extant guidelines, but also creates coherence across articles and reports, thus improving clarity (Mills‐Smith et al., ). A step further would be the supplementary sharing of all anonymized results on the participant level, thus allowing for the computations necessary for meta‐analyses, and opening the door for other types of cumulative analyses.…”

Development of infants' segmentation of words from native speech: a meta‐analytic approach

“…We have followed recent recommendations by rendering our data publicly accessible and open to updates, i.e. we have built a Community‐Augmented Meta‐Analysis (for a summary of the benefits for a research community, see Tsuji, Bergmann & Cristia, ; see also Mills‐Smith, Spangler, Panneton & Fritz, , for a discussion of the importance, in the field of infant developmental science, to incorporate effect size into our interpretations). Here, we limit ourselves to a report on the meta‐analysis; all relevant information on the database and extensive supplementary materials can be found on the companion website (http://inworddb.acristia.org).…”

Quantifying sources of variability in infancy research using the infant-directed speech preference

“…Also, there is reason to believe that effect sizes in infancy research are often incorrectly reported; for example, partial eta-squared η 2 p is often misreported as eta-squared η 2 . This confusion is likely to inflate the practical significance of QUANTIFYING VARIABILITY IN INFANT RESEARCH 10 the findings, leading to an overestimation of the statistical magnitude and importance of effects (Mills-Smith, Spangler, Panneton, & Fritz, 2015). Therefore, the mean effect size of 0.67 reported by Dunst et al (2012) is likely an overestimate of the real effect size.…”