On the importance of experimental design: pitfall traps and arthropod communities
Which pitfall traps and sampling efforts should be used to evaluate the effects of cropping systems on the taxonomic and functional composition of arthropod communities?
Recommendation: posted 04 October 2019, validated 07 October 2019
Bartomeus, I. (2019) On the importance of experimental design: pitfall traps and arthropod communities. Peer Community in Ecology, 100030. 10.24072/pci.ecology.100030
Despite the increasing refinement of statistical methods, a robust experimental design is still one of the most important cornerstones to answer ecological and evolutionary questions. However, there is a strong trade-off between a perfect design and its feasibility. A common mantra is that more data is always better, but how much is enough is complex to answer, specially when we want to capture the spatial and temporal variability of a given process. Gardarin and Valantin-Morison  make an effort to answer these questions for a practical case: How many pitfalls traps, of which type, and over which extent, do we need to detect shifts in arthropod community composition in agricultural landscapes. There is extense literature on how to approach these challenges using preliminary data in combination with simulation methods [e.g. 2], but practical cases are always welcomed to illustrate the complexity of the decisions to be made. A key challenge in this situation is the nature of simplified and patchy agricultural arthropod communities. In this context, small effect sizes are expected, but those small effects are relevant from an ecological point of view because small increases at low biodiversity may produce large gains in ecosystem functioning .
The paper shows that some variables are not important, such as the type of fluid used to fill the pitfall traps. This is good news for potential comparisons among studies using slightly different protocols. However, the bad news are that the sampling effort needed for detecting community changes is larger than the average effort currently implemented. A potential solution is to focus on Community Weighed Mean metrics (CWM; i.e. a functional descriptor of the community body size distribution) rather than on classic metrics such as species richness, as detecting changes on CWM requires a lower sampling effort and it has a clear ecological interpretation linked to ecosystem functioning.
Beyond the scope of the data presented, which is limited to a single region over two years, and hence it is hard to extrapolate to other regions and years, the big message of the paper is the need to incorporate statistical power simulations as a central piece of the ecologist's toolbox. This is challenging, especially when you face questions such as: Should I replicate over space, or over time? The recommended paper is accompanied by the statistical code used, which should facilitate this task to other researchers. Furthermore, we should be aware that some important questions in ecology are highly variable in space and time, and hence, larger sampling effort across space and time is needed to detect patterns. Larger and longer monitoring schemes require a large effort (and funding), but if we want to make relevant ecology, nobody said it would be easy.
 Gardarin, A. and Valantin-Morison, M. (2019). Which pitfall traps and sampling efforts should be used to evaluate the effects of cropping systems on the taxonomic and functional composition of arthropod communities? Zenodo, 3468920, ver. 3 peer-reviewed and recommended by PCI Ecology. doi: 10.5281/zenodo.3468920
 Johnson, P. C., Barry, S. J., Ferguson, H. M., and Müller, P. (2015). Power analysis for generalized linear mixed models in ecology and evolution. Methods in ecology and evolution, 6(2), 133-142. doi: 10.1111/2041-210X.12306
 Cardinale, B. J. et al. (2012). Biodiversity loss and its impact on humanity. Nature, 486(7401), 59-67. doi: 10.1038/nature11148
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article. The authors declared that they comply with the PCI rule of having no financial conflicts of interest in relation to the content of the article.
Evaluation round #2
DOI or URL of the preprint: https://zenodo.org/record/3451553
Version of the preprint: v2
Author's Reply, 02 Oct 2019
Decision by Ignasi Bartomeus, posted 24 Sep 2019
The authors responded to all the main criticism from the reviewers and this version reads very well. I made small edits and suggestions on the text, which can be found here: https://www.dropbox.com/s/xcwwux926564fu9/Correctedmanuscript13sept2019_IB.docx?dl=0. Some are purely editorial, and the authors are free to accept or not the proposed changes. I found several places where the sentences were a bit convoluted and I made suggestions here and there, but I am not an English native speaker, so check carefully I did not misunderstand anything. The only main comment I would like to make is to emphasize the power of simulating data based on preliminary data to assess replication needs. A quick search showed several papers proposing this approach that can be cited (e.g. https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.12306 or https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-11-94, but there may be others). If the authors provide an example of the code and data used as supplementary material, they can help other researchers to perform their own simulations based on the observed variation in other localities and/or years. This will give broader generality to the method presented. After this final small tweaks, I would be happy to recommend the preprint.
As indicated in the 'How does it work?’ section and in the code of conduct, please make sure that:
-Data are available to readers, either in the text or through an open data repository such as Zenodo (free), Dryad or some other institutional repository. Data must be reusable, thus metadata or accompanying text must carefully describe the data.
-Details on quantitative analyses (e.g., data treatment and statistical scripts in R, bioinformatic pipeline scripts, etc.) and details concerning simulations (scripts, codes) are available to readers in the text, as appendices, or through an open data repository, such as Zenodo, Dryad or some other institutional repository. The scripts or codes must be carefully described so that they can be reused.
-Details on experimental procedures are available to readers in the text or as appendices.
-Authors have no financial conflict of interest relating to the article. The article must contain a "Conflict of interest disclosure" paragraph before the reference section containing this sentence: "The authors of this preprint declare that they have no financial conflict of interest with the content of this article." If appropriate, this disclosure may be completed by a sentence indicating that some of the authors are PCI recommenders: “XXX is one of the PCI Ecology recommenders.”
In order to reach a better referencing and greater visibility of your recommended preprint, we suggest you to do the following modifications :
(i) add the following sentence in the acknowledgements: "Version 3 of this preprint has been reviewed and recommended by Peer Community In Ecology (https://doi.org/10.24072/pci.ecology.100030) »
Note that this DOI is not the DOI of your article, but the DOI of the recommendation text. The DOI of your article remains unchanged.
Doing so is very important because it would:
-indicate to readers that, unlike many other preprint in this server, your pre-print has been peer-reviewed and recommended
-make visible this information in Google Scholar search (which is quite important).
(ii) In addition, we suggest you to remove line numbering from the preprint.
==> (if you wish) we advise you to use templates (word docx template and a latex template) to format your preprint in a PCI style. This is optional. Here is the links of the templates:
Please be careful to correctly update all text in these templates (doi, authors’ names, address, title, date, recommender first name and family name …). Please be careful to also choose the badge “Open Code” if appropriate (in addition to the “Open access”, “Open data” and “Open Peer-Review” badges).
Indicate in the “cite as” box the version of the article that you are currently formatting. This should be version 3.
If some of the reviewers are anonymous, indicate for example “Ignasi Bartomeus and two anonymous reviewers”.
Evaluation round #1
DOI or URL of the preprint: https://doi.org/10.5281/zenodo.3468920
Author's Reply, 13 Sep 2019
Decision by Ignasi Bartomeus, posted 21 Feb 2019
After carefully reading the manuscript and reading the reviewers comments, I think that analyzing how to improve a common sampling technique used for describing ground arthropod communities can constitute a good contribution to the field. However, I concur with both reviewers that the manuscript needs to be presented in a more clear way, and acknowledge better its limitations. I made a number of wording suggestions in the text to improve clarity, especially about the results presented (see the document here: https://www.dropbox.com/s/3qyv35ljifj1yfc/Manuscript_IB.docx?dl=0). My only main concern is regarding the simulation (see reviewer 2 detailed advise). I am not sure the data at hand allows testing the sampling effort question, but if the authors think so, it should be clearly justified in the paper. Clear recommendations on what pitfall traps are optimal in different conditions would be of great help. I hope our comments are helpful to strengthen the manuscript.
Best, Ignasi Bartomeus