simstudy: Illuminating research methods through data generation

Goldfeld, Keith; Wujciak-Jens, Jacob

doi:10.21105/joss.02763

Cited by 34 publications

(23 citation statements)

References 15 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We used the R packages simstudy [38] (version 0. . Each older brother increased these odds by a factor of (θ + 1) with @ℎB@C ∈ (-1,∞).…”

Section: Methodsmentioning

confidence: 99%

The Fraternal Birth-Order Effect as Statistical Artefact: Convergent Evidence from Probability Calculus, Simulated Data, and Multiverse Meta- Analysis

Jk¹,

Kossmeier²,

Voracek³

et al. 2021

Preprint

View full text Add to dashboard Cite

For a quarter of a century researchers investigating the origins of sexual orientation have largely ascribed to the fraternal birth order effect (FBOE) as a fact, holding that older brothers increase the odds of homosexual orientation among men through an immunoreactivity process. Here, we triangulate the empirical foundations of the FBOE from three distinct, informative perspectives: First, drawing on basic probability calculus, we deduce mathematically that the body of statistical evidence of the FBOE rests on the false assumptions that effects of family size should be controlled for and that this could be achieved through the use of ratio variables. Second, using a data-simulation approach, we demonstrate that by using ratio variables, researchers are bound to falsely declare corroborating evidence of an excess of older brothers at a rate of up to 100%, and that valid approaches attempting to quantify a potential excess of older brothers among homosexual men must control for the confounding effects of the number of older siblings. And third, we re-examine the empirical evidence of the FBOE by using a novel specification-curve and multiverse approach to meta-analysis. This yielded highly inconsistent and moreover similarly-sized effects across 64 male and 17 female samples (N = 2,778,998), compatible with an excess as well as with a lack of older brothers in both groups, thus, suggesting that almost no variation in the number of older brothers in men is attributable to sexual orientation.

show abstract

“…We used the R packages simstudy [38] (version 0. . Each older brother increased these odds by a factor of (θ + 1) with @ℎB@C ∈ (-1,∞).…”

Section: Methodsmentioning

confidence: 99%

The Fraternal Birth-Order Effect as Statistical Artefact: Convergent Evidence from Probability Calculus, Simulated Data, and Multiverse Meta- Analysis

Jk¹,

Kossmeier²,

Voracek³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The generation procedure is simple: We generate normally-distributed points using the R library (Goldfeld & Wujciak-Jens, 2020 ), by providing at the level of the study the number of data points to be generated (number of infants), the average preference quotient for the first and second day, as well as their standard deviations, and the test-retest correlation coefficient. We then filter the resulting generated dataset to only keep it when the parameters of the generated data are sufficiently close to the intended ones (e.g., when the correlation coefficient of the generated data points is within .01 from the intended r ).…”

Section: The Present Studymentioning

confidence: 99%

“…In this experiment, we generate synthetic data for 10 simulated studies, each consisting of 100 participants, using the R library (Goldfeld & Wujciak-Jens, 2020 ). We systematically and independently vary (i) The preference quotient for the first day’s test PQ1 (which determines our effect size Cohen’s d for PQ1) and that for the second day PQ2, (ii) The test-retest correlation in performance across days r .…”

Section: Experiments 2: Synthetic Datamentioning

confidence: 99%

Toward Cumulative Cognitive Science: A Comparison of Meta-Analysis, Mega-Analysis, and Hybrid Approaches

Koile

Cristià

2021

Open Mind

View full text Add to dashboard Cite

There is increasing interest in cumulative approaches to science, in which instead of analyzing the results of individual papers separately, we integrate information qualitatively or quantitatively. One such approach is meta-analysis, which has over 50 years of literature supporting its usefulness, and is becoming more common in cognitive science. However, changes in technical possibilities by the widespread use of Python and R make it easier to fit more complex models, and even simulate missing data. Here we recommend the use of mega-analyses (based on the aggregation of data sets collected by independent researchers) and hybrid meta- mega-analytic approaches, for cases where raw data is available for some studies. We illustrate the three approaches using a rich test-retest data set of infants’ speech processing as well as synthetic data. We discuss advantages and disadvantages of the three approaches from the viewpoint of a cognitive scientists contemplating their use, and limitations of this article, to be addressed in future work.

show abstract

“…The number has also good properties for the generation of the time points. For simulation of the expression data, we used the statistical programming language R 3.6 and the R package simstudy [23]. For each time point, we first generated three data points sampled from a normal distribution with a mean of zero and a variance of 5, the mother effects.…”

Section: Artificial Expression Datamentioning

confidence: 99%

Change Point Detection For Clustered Expression Data In Short Time Series.

Sieg¹,

Sciesielski²,

Kirschner³

et al. 2021

Preprint

View full text Add to dashboard Cite

Background: In longitudinal studies, observations are made over time. Hence, the single observations at each time point are dependent, making them a repeated measurement. In this work, we explore a different, counterintuitive setting: At each developmental time point, a lethal observation is performed on the pregnant or nursing mother. Therefore, the single time points are independent. Furthermore, the observation in the offspring at each time point is correlated with each other because each litter consists of several (genetically linked) littermates. In addition, the observed time series is short from a statistical perspective as animal ethics prevent killing more mother mice than absolutely necessary, and murine development is short anyway. We solve these challenges by using multiple contrast tests and visualizing the change point by the use of confidence intervals.Results: We used linear mixed models to model the variability of the mother. The estimates from the linear mixed model are then used in multiple contrast tests.There are a variety of contrasts and intuitively, we would use the Changepoint method. However, it does not deliver satisfying results. Interestingly, we found two other contrasts, both capable of answering different research questions in change point detection: i) Should a single point with change direction be found, or ii) Should the overall progression be determined? The Sequen contrast answers the first, the McDermott the second. Confidence intervals deliver effect estimates for the strength of the potential change point. Therefore, the scientist can define a biologically relevant limit of change depending on the research question.Conclusion: We present a solution with effect estimates for short independent time series with observations nested at a given time point. Multiple contrast tests produce confidence intervals, which allow determining the position of change points or to visualize the expression course over time. We suggest to use McDermott’s method to determine if there is an overall significant change within the time frame, while Sequen is better in determining specific change points. In addition, we offer a short formula for the estimation of the maximal length of the time series.

show abstract

simstudy: Illuminating research methods through data generation

Cited by 34 publications

References 15 publications

The Fraternal Birth-Order Effect as Statistical Artefact: Convergent Evidence from Probability Calculus, Simulated Data, and Multiverse Meta- Analysis

The Fraternal Birth-Order Effect as Statistical Artefact: Convergent Evidence from Probability Calculus, Simulated Data, and Multiverse Meta- Analysis

Toward Cumulative Cognitive Science: A Comparison of Meta-Analysis, Mega-Analysis, and Hybrid Approaches

Change Point Detection For Clustered Expression Data In Short Time Series.

Contact Info

Product

Resources

About