We thank our colleagues for their constructive comments that relate to our two articles in the July 2017 issue of BJSM.
Bermon and Garnier StudyThe main criticisms expressed about the data used and our statistical analysis were (1) our 'concentration' on free testosterone (fT) rather than total testosterone (T) 3 (presumably because the T results were only presented in the Internet version of the paper), (2) the fact that 17.3% of the athletes were sampled at both World Championships (Daegu and Moscow), 3 (3) the fact that no correlation analysis was performed other than comparison of fT or T tertiles, 3 (4) the absence of statistical comparison between a group with high T levels and a group with normal T levels 4 and (5) the lack of adjustment for multiple comparisons (suggesting that the significant differences observed in our study could have happened by chance).
5Taking this last criticism first, we note that we presented an exploratory study, without no attempt to claim confirmatory results. In fact, the exploratory evidence presented in the study is strong, and correction for multiplicity may be too conservative. But we agree that the results should be put into context. At the type 1 error level of 0.05, we could expect 1 of 20 hypotheses tested to be significant merely by chance, that is, p <0.05. In this study, we have observed significant correlations at the 0.05 level in five events, out of 21 events in total. Therefore, it is very unlikely that all these findings are caused by chance. Moreover, the five flagged events were showing similar findings in both fT and T, which indicates that the evidence and findings are robust.Franklin et al report that, using Fisher's combination test, they were unable to reject the hypothesis that the p values calculated from the published data were not inconsistent with there being no advantage to high fT women in any of the five events, 5 but they do not explain their analysis in any detail or support their conclusion with any evidence. Their suggestion to conduct one single test of correlation across all 21 events combined is inappropriate because the hypothesis is that elevated testosterone levels enhance performance in certain events but not others. Combining results from the events of long sprint to middle-distance running shows with additional statistical calculation a p value of 0.003 for the correlation between testosterone levels and competition results, which is very strong statistical evidence for a true correlation not caused by chance.
additional SenSitivity analySeSTo address the other criticisms, we have now performed a sensitivity analysis using a modified data set in which (1) observations from athletes who participated in both World Championships were only counted once, that is, the first observed value, and (2) only T concentrations were used (not fT). As for aggregating results, we used running events from 400 m up to 1 mile, on the basis that that is where T produces its greatest performance-enhancing effects (by increasing lean body mass and the...