A stronger statistical test of stream restoration experiments

Karl Cottenie based on reviews by Eric Harvey and Mariana Perez Rocha

A recommendation of:
David Murray-Stoker. On the efficacy of restoration in stream networks: comments, critiques, and prospective recommendations (2020), bioRxiv, 611939, ver. 7 peer-reviewed by Peer Community in Ecology. 10.1101/611939
Submitted: 21 September 2019, Recommended: 03 May 2020
Cite this recommendation as:
Karl Cottenie (2020) A stronger statistical test of stream restoration experiments. Peer Community in Ecology, 100052. 10.24072/pci.ecology.100052

The metacommunity framework acknowledges that local sites are connected to other sites through dispersal, and that these connectivity patterns can influence local dynamics [1]. This framework is slowly moving from a framework that guides fundamental research to being actively applied in for instance a conservation context (e.g. [2]). Swan and Brown [3,4] analyzed the results of a suite of experimental manipulations in headwater and mainstem streams on invertebrate community structure in the context of the metacommunity concept. This was an important contribution to conservation ecology.
However, David Murray-Stoker [5] was not satisfied with their statistical analyses, and recreated, and more importantly, improved their original analyses in the peer-reviewed article. The new analyses are based on a combination of a more consistent site selection, checking the model assumptions, using different estimation procedures, and focusing more on effect size calculations versus statistical significance. This peer-reviewed article is thus the perfect example of the advantages of open research: the original authors making available both the data and their R script files, initially first updating the analyses and results themselves, followed by more in-depth analyses of the original data and question.
This peer reviewed went through a very in-depth process itself, with several rounds of questions and feedback that addressed both the statistical analyses, the interpretation of the results, and the conclusions. It also, however, addressed something that is often harder to provide feedback on, for instance the tone of the argument. I hope that scientists interested in these issues will not only read the final manuscript, but also the different steps of the peer review processes. These are very informative, I think, and provide a more complete picture of mainly the raison for certain decisions.
Not only does this provide the reader interested in stream conservation with the opportunity to make up their own mind on the appropriateness of these decisions, but it could potentially lead to more analyses of this important data set. For instance, maybe a formal meta-analysis that starts with the effect sizes of all the original studies might bring some new insights into this question?

References

[1] Leibold, M. A., Holyoak, M., Mouquet, N. et al. (2004). The metacommunity concept: a framework for multi‐scale community ecology. Ecology letters, 7(7), 601-613. doi: 10.1111/j.1461-0248.2004.00608.x
[2] Heino, J. (2013). The importance of metacommunity ecology for environmental assessment research in the freshwater realm. Biological Reviews, 88(1), 166-178. doi: 10.1111/j.1469-185X.2012.00244.x
[3] Swan, C. M., and Brown, B. L. (2017). Metacommunity theory meets restoration: isolation may mediate how ecological communities respond to stream restoration. Ecological Applications, 27(7), 2209-2219. doi: 10.1002/eap.1602
[4] Swan, C. M., and Brown, B. L. (2018). Erratum for: Metacommunity theory meets restoration: isolation may mediate how ecological communities respond to stream restoration. Ecological Applications 28:1370–1371. doi: 10.1002/eap.1738
[5] Murray-Stoker, D. (2020). On the efficacy of restoration in stream networks: comments, critiques, and prospective recommendations. bioRxiv, 611939, ver. 7 peer-reviewed and recommended by PCI Ecology. doi: 10.1101/611939