The evolutionarily stable strategy (or ESS; also evolutionary stable strategy) is a central concept in game theory introduced by John Maynard Smith and George R. Price in 1973 (a full account is given by Maynard Smith, 1982). It is based on a concept of a population of organisms playing a certain strategy, that a mutant allele that causes organisms to adopt a different strategy cannot invade the population, but will instead be selected out by natural selection.
The concept was based on W.D. Hamilton's (1967) unbeatable strategy; the difference is that an unbeatable strategy is resistant to large migrations of different strategies. Through Hamilton's work on sex ratios the concept can be traced back through Ronald Fisher (1930) and Charles Darwin (1859) (see Edwards, 1998).
An ESS depends on the idea of invasion, where a population of strategy-X players is visited by a strategy-Y player. The new player is said to invade if, following strategy Y, he scores better than the average strategy-X player. Assuming players are able to choose and switch strategies, this would induce the indigenous population to start switching to strategy Y. In many cases there are diminishing returns for the later adopters, and what follows is an equilibrium ratio of strategy-X players to strategy-Y players.
A strategy X is evolutionarily stable if there is no strategy Y that can invade it. That is, anybody bringing a new strategy into a population of strategy-X players will fare no better on average than the X players are already doing. (See the closely-related Nash equilibrium.) ESS is stable in respect to randomly and occasionally occurring invading strategies, thus it is not stable in respect to mass counts of invaders.
The recent, controversial sciences of sociobiology and now evolutionary psychology attempt to explain animal and human behavior and social structures, largely in terms of evolutionarily stable strategies. For example, in one well-known 1995 paper (http://www.bbsonline.org/Preprints/OldArchive/bbs.mealey.html) by Linda Mealey, sociopathy (chronic antisocial/criminal behavior) is explained as a combination of two such strategies.
In order for a strategy to be evolutionarilystable, it must have the property that if almost every member of the population follows it, no mutant (that is, an individual who adopts a novel strategy) can successfully invade.
Roughly, if only two pure strategies exist, then given a (possibly mixed) evolutionarilystablestrategy, the corresponding state of the population is a stable state under the replicator dynamics.
This representation of strategy selection clearly presupposes hyperrational players and fails to represent the process by which one player observes his opponent's behavior, learns from these observations, and makes the best move in response to what he has learned (as one might expect, for there is no need to model learning in hyperrational individuals).
The difference between a Nash equilibrium and an ESS is that a Nash equilibrium may sometimes exist due to the assumption that rational foresight prevents players from playing an alternative strategy with no short term cost, but which will eventually be beaten by a third strategy.
An evolutionarilystable state is a dynamical property of a population to return to using a strategy, or mix of strategies, if it is perturbed from that strategy, or mix of strategies.
ESS ยท Risk dominance In game theory, the Nash equilibrium (named after John Forbes Nash, who proposed it) is a kind of solution concept of a game involving two or more players, where no player has anything to gain by changing only his or her own strategy unilaterally.