How to identify strategy use and adaptive strategy selection: The crucial role of chance correction in weighted compensatory strategies

Abstract

Model comparisons based on choices are a common method to assess adaptive strategy selection models. We show that one common methodological choice prevailing in parts of this work makes it hard to draw sound conclusions concerning the central questions of how often people indeed use simplified strategies (e.g., fast-and-frugal heuristics) and whether they select between them adaptively. Conclusions from this work are biased insofar that more complex weighted compensatory strategies, such as Franklin’s Rule, that are considered alternatives to fast-and-frugal heuristics are specified in a way that leads to a systematic underestimation of the degree to which they are relied on. Specifically, the common practice of not correcting cue validities for chance level results in a substantial disadvantage for weighted compensatory strategies. Model recovery simulations and a reanalysis of published data correspondingly show overestimation of the reliance on fast-and-frugal heuristics for various environments used in prominent studies. Furthermore, our results indicate that the magnitude of this overestimation is contingent on the structure of the environment so that strategy selection is mimicked without any material basis. As a result, even completely rational individuals would be erroneously classified as adaptive users of simple strategies. We recommend that chance corrected implementations of weighted compensatory strategies should be included in all future studies investigating the usage of adaptive strategies and argue that comprehensive re-analyses of previous work are warranted.