Generalizing research findings for enhanced reproducibility: an approach based on verbal alternative representations

Kenett, Ron S.; Rubinstein, Abraham

doi:10.1007/s11192-021-03914-1

Cited by 22 publications

(11 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This approach generates a table of alternative representations with a boundary of meaning (BOM). 26,29 The BOM is a demarcation line between claims, presented in alternative ways, and seemingly similar representations of findings not supported by With this context, we formulate questions for reviewers of statistical analysis in applied research as checklists. These are listed in Table 1.…”

Section: Statistical Analysis Of Applied Researchmentioning

confidence: 99%

“…Alternatives with surface similarity seem similar to the target conceptual statement but have different meaning. This approach generates a table of alternative representations with a boundary of meaning (BOM) 26,29 . The BOM is a demarcation line between claims, presented in alternative ways, and seemingly similar representations of findings not supported by the research.…”

Section: Statistical Analysis Of Applied Researchmentioning

confidence: 99%

“…Finally, findings have to be presented and generalized. Generalization can be achieved by a range of methods, some intuitive, some conceptual and some more formal, invoking, for example, causal arguments 25,26 . Findings can be presented in different ways.…”

Section: Statistical Analysis Of Applied Researchmentioning

confidence: 99%

“…Findings can be presented in different ways. One approach is based on alternative verbal representations, some with meaning equivalence and some with surface similarity 26 . Verbal expression of research findings has been proposed in Greenland 27 and Yarkoni 28 .…”

Section: Statistical Analysis Of Applied Researchmentioning

confidence: 99%

See 3 more Smart Citations

Helping reviewers assess statistical analysis: A case study from analytic methods

Kenett

Francq²

2022

Analytical Science Advances

Self Cite

View full text Add to dashboard Cite

Analytic methods development, like many other disciplines, relies on experimentation and data analysis. Determining the contribution of a paper or report on a study incorporating data analysis is typically left to the reviewer's experience and good sense, without reliance on structured guidelines. This is amplified by the growing role of machine learning driven analysis, where results are based on computer intensive algorithm applications. The evaluation of a predictive model where cross validation was used to fit its parameters adds challenges to the evaluation of regression models, where the estimates can be easily reproduced. This lack of structure to support reviews increases uncertainty and variability in reviews. In this paper, aspects of statistical assessment are considered. We provide checklists for reviewers of applied statistics work with a focus on analytic method development. The checklist covers six aspects relevant to a review of statistical analysis, namely: (1) study design, (2) algorithmic and inferential methods in frequentism analysis, (3) Bayesian methods in Bayesian analysis (if relevant), (4) selective inference aspects, (5) severe testing properties and (6) presentation of findings. We provide a brief overview of these elements providing references for a more elaborate treatment. The robustness analysis of an analytical method is used to illustrate how an improvement can be achieved in response to questions in the checklist. The paper is aimed at both engineers and seasoned researchers.

show abstract

Section: Statistical Analysis Of Applied Researchmentioning

confidence: 99%

Section: Statistical Analysis Of Applied Researchmentioning

confidence: 99%

Section: Statistical Analysis Of Applied Researchmentioning

confidence: 99%

Section: Statistical Analysis Of Applied Researchmentioning

confidence: 99%

See 2 more Smart Citations

Helping reviewers assess statistical analysis: A case study from analytic methods

Kenett

Francq²

2022

Analytical Science Advances

Self Cite

View full text Add to dashboard Cite

show abstract

“…Statistics should offer methods for assessing such BOMs. In Kenett and Rubinstein we propose using a sign type error proposed by Gelman and Carlin to do that 2,3 …”

Section: Beyond P < 005mentioning

confidence: 99%