Whither scientific debate? A rebuttal of “Contextualising UK moorland burning studies: geographical versus potential sponsorship-bias effects on research conclusions” by Brown and Holden (bioRxiv 2019; 731117)

Abstract

To read the preprint which this publication seeks to criticise, see here:https://www.biorxiv.org/content/10.1101/731117v1To read our original peer-reviewed critique of the EMBER project, see here:https://besjournals.onlinelibrary.wiley.com/doi/abs/10.1111/1365-2664.134761. We recently published a peer-reviewed critique of the EMBER report. In a preprint response, Brown & Holden (2019) resorted to making spurious accusations of undeclared competing interests, a series of disingenuous arguments about the robustness of the EMBER results, as well as false claims of sponsorship bias. We feel that much of what they wrote falls well outside the realm of respectable scientific debate.2. Crucially, however, Brown & Holden (2019) did not address our previous criticisms by providing a robust reanalysis of the EMBER report data that correctly accounted for site and covariate effects within the same statistical model. In our reply, we also present additional flaws which further call into question the EMBER results.3. Brown & Holden (2019) also produced a literature review to show that the EMBER results are not out of line with the broader evidence base. However, they included papers not directly relevant to the EMBER report we criticised. Therefore, we have carried out a more accurate review. Our results indicate that the quantity and quality of available literature make it difficult to contextualise the findings of the EMBER report.4. Finally, Brown & Holden (2019) present an error-stricken analysis of grouse moor sponsorship bias within the prescribed burning literature. Their claim that grouse moor funded research “should be treated with extreme caution by the policy community” goes well beyond what their data allows them to say. Not only does such a claim egregiously impugn the reputation of many scientists in the field, but it also contradicts the long-established notion that it is the quality of the science that should drive evidence-led policy. 5. Policy Implications. The results of the EMBER report remain unreliable. Therefore, for the time being, it should not be considered as valid evidence by policymakers. We suggest that the data from the EMBER report is reanalysed to address the shortfalls that we identify. Only then can the EMBER data and findings be used to inform upland land management policy. Also, to provide clarity to policymakers, we recommend that an independent audit into evidence reliability is carried out across the prescribed burning evidence base.

Prescribed Fire in UK Heather-Dominated Blanket Bog Peatlands: A Critical Review of “Carbon Storage and Sequestration by Habitat: A Review of the Evidence (Second Edition)” by Gregg et al., 2021

“…As such, the significance values reported are unreliable, likely too low and should not be used to generalise [117]; iv. Peat surface temperature results are suggestive of measurement error [7,8], and effect sizes remain unknown (but according to the published graphs [116], are likely to be small).…”

“…We have outlined, in detail, the methodological and statistical issues with the EM-BER project [7,8], but others have pointed out related limitations in the experimental design [84]. Crucially, the EMBER authors failed to address these issues in their replies to our criticisms [9,10].…”

“…In the UK, prescribed fire is primarily used for grouse moor management. Consequently, it has become a contentious and controversial topic of debate in the public, academic and policymaking arena [2][3][4][5][6][7][8][9][10]. In contrast, outside the UK, prescribed fire is considered a valuable habitat and vegetation management tool, especially for reducing fuel loads and, thereby, wildfire risk and damage [3][4][5][11][12][13].…”

Vegetation fires in temperate upland heaths: environmental impacts, recovery, and management implications

“…Despite these efforts to ensure conditions were conducive to comparable sampling areas and fire events, it is acknowledged that the results of space-for-time studies are not as reliable or accurate as those produced through controlled experimentation as some differences in site histories and burn characteristics are inevitable (França et al, 2016). It is also important to acknowledge that these limitations do not preclude studies of this kind from providing useful insight if conducted appropriately and transparently (Ashby and Heinemeyer, 2019b). In this study, a time-series or chronosequence of soil recovery cannot be produced due to the lack of burn treatment replication (e.g.…”

“…Catchment-scale studies addressing the extent to which changes propagate downstream are highly beneficial to assess the broader impacts of burning on water colour and DOC. It is, however, vitally important when conducting these kinds of studies to ensure any paired "burnt" and "unburnt" catchments are sufficiently comparable to ensure a reliable interpretation of results (Ashby and Heinemeyer, 2019b;.…”