Assessing the reliability of the methods we use in actually measuring the intended trait should be one of our first priorities when designing a study – especially when the trait in question is not directly observable and is measured through a proxy. 

This is the case for cognitive traits, which are often quantified through measures of behavioral performance. Behavioral flexibility is of particular interest in the context of great environmental changes that a lot of populations have to experiment. This type of behavioral performance is often measured through reversal learning experiments (Bond 2007). In these experiments, individuals first learn a preference, for example for an object of a certain type of form or color, associated with a reward such as food. The characteristics of the rewarded object then change, and the individuals hence have to learn these new characteristics (to get the reward). The time needed by the individual to make this change in preference has been considered a measure of behavioral flexibility.

Although reversal learning experiments have been widely used, their construct validity to assess behavioral flexibility has not been thoroughly tested. This was the aim of McCune and collaborators' (2023) study, through the test of the repeatability of individual performance within and across contexts of reversal learning, in the great-tailed grackle.

This manuscript presents a post-study of the preregistered study* (Logan et al. 2019) that was peer-reviewed and received an In Principle Recommendation for PCI Ecology (Coulon 2019; the initial preregistration was split into 3 post-studies).
Using 34 great-tailed grackles wild-caught in Tempe, Arizona (USA), the authors tested in aviaries 2 hypotheses:

• First, that the behavioral flexibility measured by reversal learning is repeatable within individuals across sessions of the same experiment;
• Second, that there is repeatability of the measured behavioral flexibility (within individuals) across different types of reversal learning experiments (context).

The first hypothesis was tested by measuring the repeatability of the time needed by individuals to switch color preference in a color reversal learning task (colored tubes), over serial sessions of this task. The second one was tested by measuring the time needed by individuals to switch solutions, within 3 different contexts: (1) colored tubes, (2) plastic and (3) wooden multi-access boxes involving several ways to access food.

Despite limited sample sizes, the results of these experiments suggest that there is both temporal and contextual repeatability of behavioral flexibility performance of great-tailed grackles, as measured by reversal learning experiments.

Those results are a first indication of the construct validity of reversal learning experiments to assess behavioral flexibility. As highlighted by McCune and collaborators, it is now necessary to assess the discriminant validity of these experiments, i.e. checking that a different performance is obtained with tasks (experiments) that are supposed to measure different cognitive abilities.
 
* A pre-registered study is a study in which context, aims, hypotheses and methodologies have been written down as an empirical paper, peer-reviewed and pre-accepted before research is undertaken. Pre-registrations are intended to reduce publication bias and reporting bias.
 
REFERENCES
 
Bond, A. B., Kamil, A. C., & Balda, R. P. (2007). Serial reversal learning and the evolution of behavioral
flexibility in three species of north american corvids (Gymnorhinus cyanocephalus, Nucifraga columbiana,
Aphelocoma californica). Journal of Comparative Psychology, 121 (4), 372. https://doi.org/10.1037/0735-7036.121.4.372

Coulon, A. (2019) Can context changes improve behavioral flexibility? Towards a better understanding of species adaptability to environmental changes. Peer Community in Ecology, 100019. https://doi.org/10.24072/pci.ecology.100019

Logan, CJ, Lukas D, Bergeron L, Folsom M, & McCune, K. (2019).  Is behavioral flexibility related to foraging and social behavior in a rapidly expanding species? In Principle Acceptance by PCI Ecology of the Version on 6 Aug 2019. http://corinalogan.com/Preregistrations/g_flexmanip.html

McCune KB, Blaisdell AP, Johnson-Ulrich Z, Lukas D, MacPherson M, Seitz BM, Sevchik A, Logan CJ (2023) Using repeatability of performance within and across contexts to validate measures of behavioral flexibility. EcoEvoRxiv, ver. 5 peer-reviewed and recommended by Peer Community in Ecology. https://doi.org/10.32942/X2R59K