Game Theory

ECON 159 - Lecture 11 - Evolutionary Stability: Cooperation, Mutation, and Equilibrium

Chapter 1. Game Theory and Evolution: Evolutionarily Stable Strategies - Example [00:00:00]

Professor Ben Polak: So we're going to do a new topic today, we're going to talk about evolution. We're going to talk about the connection between evolution and Game Theory. Now there's going to be an extra reading--I've already emailed you about this--there's going to be an extra reading on this for the people who want it. I'm going to--There's a reading packet that's available to anybody who wants it. I emailed you all, the website where you can order that reading packet and figure out where to pick it up. It's not compulsory. You don't have to look at that reading packet. It's just that it might help. In addition, as with last Wednesday, since some of the material here is new, I have written a handout that goes with this lecture and that handout will appear magically this afternoon on the website. So if things today are fast, don't worry there's a handout that goes with it.

All right, so why look at evolution in the context of Game Theory? There are really two reasons. The first reason is because of the influence of Game Theory on biology. It turns out that in the last few decades, there's been an enormous amount of work done in biology, in particular, looking at animal behavior and using Game Theory to analyze that animal behavior. And just to give you a loose idea about how this works, the idea is to relate strategies with genes, or at least a phenotype of those genes, and to relate payoffs to genetic fitness. So the idea is that strategies are related to genes and the payoffs in the games are related to genetic fitness. And the big idea of course, is that strategies grow if they do well. So strategies that do well in these games grow. Strategies that do less well die out. So we're going to be visiting that today.

One thing to bear in mind from the start is that there's an important difference between Game Theory as analyzed in animal behavior and biology, and Game Theory that we've been doing so far in the class. And that is, that we're going to think of behaviors, these strategies played by animals, not as chosen by reasoning individuals but rather as being hardwired. So if strategies grow it isn't that some lion or ant has chosen that strategy. It's simply that the ant or lion who has that gene that corresponds to that strategy grows and has many children. So it's important, this is a new idea for us, that these strategies are hardwired.

So it's interesting, in and of itself, but there's also an important influence the other way. So a second reason for studying this stuff is that there's been an enormous influence from biology or from evolutionary biology in particular, back on the social sciences.

This influence the other way uses evolution largely as a metaphor. And you'll find this if you're a political scientist, if you're a historian, if you're an anthropologist, if you're a sociologist. Let me give you an example from Economics, since it's perhaps closer to home. So here's an example. You might imagine some firms in the marketplace and you could think of these firms not necessarily reasoning out what is the most profitable strategy for them, or what is the most cost-reducing strategy for them. However, they may just have rules of thumb to select their strategies. However, in a competitive marketplace, survival of the fittest firms will lead to us ending up with a bunch firms who have low costs and high profits. That make sense? So here competition in the marketplace substitutes for competition in the jungle, as it were. The analog of a gene dying out is a firm going bankrupt.

So this is the kind of ideas we're going to explore. These ideas are going to the background for today and Monday. Now before I go on, I want to put in a couple of kind of important riders. So there are two things that I am not. The first thing I am not is I'm not a biologist. I'm sure there are people in this room who are biologists who know a lot more about the genetic side of this than I do. So I'm not going to attempt to teach you biology here, I'm going to focus on the Game Theory part. The second thing I'm not is I'm not an American citizen. I worry slightly when we talk about evolution because I realize it's very controversial here, and some of you might be from places like say Kansas. And if you are from someplace like Kansas, then please don't write to your Senator about this, or if they see the movie, just tell them it was some other English guy giving the lecture.

So what we're going to actually do is we're going to look at a highly simplified model of evolution, at least for today. We might get onto a more complicated model on Monday, but today it's going to be very highly stylized. So this is going to be our extremely simplified model. This simplified model is going to do tremendous violence to the biology, but it's really just to fix some ideas. So we're going to focus on what we're going to call within species competition. So the lions are competing against the lions and the ants are competing against the ants. The way we're going to think about this is we're going to look at symmetric two-player games. So we look at very simple games, they're only going to involve two players, and they're going to be symmetric games, which means both players have the same strategies, both players have the same payoffs. The way we're going to think of is, we're going to imagine that there's a large population out there, each of whom is playing a particular strategy, and we're going to assume that what happens is, we randomly pick two people from that population and pair them up.

So everyone in this large population will be randomly paired with someone else, they'll play the strategy that they are hardwired to play, and then we'll see what happens. So the idea here is there's a large population of, if you like animals, which are hardwired to play particular strategies, and we're going to have random matching. The idea here is when we do lots of these random matching, we're going to keep track of the average payoffs. So what we're going to focus on are the average payoffs of particular strategies when randomly matched in these games. Again, the underlying idea is that relatively successful strategies will grow and relatively unsuccessful ones will decline, I'm not going to write that, that's obviously the other part of that. So relatively successful strategies are growing and relatively unsuccessful strategies are declining.

Now to keep things simple, I'm not going to do any dynamics here, that would take us beyond the math that you could probably, that a lot of you can do in the class. So we're not going to worry about dynamics here, but this is the underlying dynamic. The idea is that if a strategy is successful that strategy will grow in the population, if a strategy is unsuccessful it will decline. So this isn't horrible violence to biology yet, it's the next bit that's the horrible violence. So to keep things simple today we're going to assume that there is no gene redistribution. So in principle what we're looking at is asexual reproduction, which is clearly not a good model but it will do for today. So you want to think about a practical example of asexual reproduction, think about root vegetables or judging from our experiments with the dating game, we better hope that's true for Economics majors as well. Maybe the fact that I'm focusing on asexual reproduction will get me off with the guys in Kansas: maybe I won't lose my green card after all.

So that's going to be our basic story, and the basic idea we're going to use is this. It's an idea due to a guy called Maynard Smith in the 70s, although obviously the big idea goes back earlier. So this is the big idea. Suppose that we imagine that there's a particular game and there's these large populations--think of yourselves as the large population--and suppose that the entire population were all playing the same strategy, call it S, so they're all hardwired to play the same strategy S. Suppose now that there's a mutation, so some small group start playing some other strategy, let's call it S'. What we want to ask is will that small mutation group, the S' strategy, will it thrive or will it die out? If it's true that, for all possible mutations, all the possible little groups of mutation of people who are playing S', they'll die out, then we'll say that the original strategy, the strategy S, is evolutionarily stable. We're going to write that out more formally later on but that's just the basic idea, so let's just say it again.

There's a strategy out there that everyone's playing, call it S. We're going to look at a mutation, that's S'. So a small group of people are going to start playing S'. They're going to go on being randomly matched, everyone's going to be randomly matched, and we're going to ask if the S' group does well, in which case they'll grow, or does badly, in which case they'll shrink and eventually die out. If they die out we'll say that S was evolutionarily stable, that's true for all possible mutations. Just notice when we're randomly matching them, one thing to note is since there's only a small mutation to start with, most of the time when they're randomly matched, they're going to match against somebody who's still playing S. Occasionally they're going to meet one of the other mutants, but most of the time we're going to have to worry about how the mutants do against the incumbent population.

That was too abstract to really get one's head around, so let's try and do an example. So let's remove our motivation and get down to actually doing some work. So we're going to start with a very simple example that you'll all recognize. Here's a game, it's a two-by-two game, and the payoffs are as follows, (2, 2), (0, 3), (3, 0) and (1, 1). We'll call these strategies cooperate or defect: C for cooperate, D for defect. You'll all recognize immediately that this game is what? This is Prisoners' Dilemma. So we're going to start by imagining these animals playing Prisoner's Dilemma. And to put it into context, imagine that these are a group of lions--and again leave aside the fact that it's asexual reproduction for a second--imagine these are a group of lions. And cooperating means cooperating on the hunt, it means using a lot of energy going after--as you cooperate in a group while hunting. And defecting here would mean not working hard on the hunt and letting the other lions catch the antelope or whatever, and then just sharing in the spoils: so "free riding" basically.

Or another example, think about ants with an ant nest, imagine this ant nest has been attacked by I don't know what, some other creature. You could imagine that cooperating is joining in, in defending the nest, at the risk of being hurt and defecting is running away. We all know in this game roughly how to analyze it. I'm not going to go over that now. I want to ask a different question today. I want to ask in this model of asexual reproduction is cooperation evolutionarily stable? So to try and illustrate that, let's just think about this game being played for real out there. So what I'm going to do is we're going to start off by imagining that you are all, I don't know, ants, lions, what do you want to be? Ants I think. So you're, from up here you look like ants. You're all ants and all of you have been hardwired to play the strategy C. So life's going on fine and let's do some random matching here, so suppose that we randomly match this ant, so stand up a second, so this ant gets randomly matched with this ant, so they now play this Prisoner's Dilemma against one another, but both of them have been hardwired to cooperate, so they both cooperate, is that right? So this ant whose name is?

Student: Lenore.

Professor Ben Polak: Lenore the ant cooperates and-

Student: Woo.

Professor Ben Polak: - and Woo the ant cooperates. So since they both cooperate there, we can look at what their payoff is, their payoff is cooperate against cooperate, so they get 2, so that's pretty good, so they're doing pretty well in terms of genetic fitness and suddenly there are two ants, right? So this ant produces another ant, someone without a computer, and this ant produces another ant, so they're doing fine and the whole population is matching against other cooperative ants, and more ants are being produced and everything is fine and dandy, everything looks good. Sit down a second. Now imagine that there's a mutation, a small mutation, and this small mutation involves a group of rather nasty uncooperative ants, so a scary mutation. So here's our scary mutation. Let's use the T.A.'s. So the T.A.'s are my scary mutations, so all of the T.A.'s here are the scary mutations, there's another one over there. So there's a bunch, everyone else is still playing cooperate, there's this vast number of cooperative ants, but now there's these small number of ants with this mutation that says don't cooperate, play D.

So let's see what happens with random matching. So a lot of these cooperative ants are still matching with each other and doing fine, but that's not the point. What we're worried about is what's going to happen to the mutants? So let's pick our representative nasty mutant, so that's going to be Rahul, so Rahul is our scary mutant and let's go back and randomly match him against one of our incumbent ants, our nice, cooperative ants, so let's pick this one. So it just happens that he's randomly matched with this cooperative ant, and this cooperative ant, whose name is?

Student: Nick.

Professor Ben Polak: Nick is hardwired since he's a regular ant he's hardwired to play cooperate, so he's going to play cooperate. Say cooperate, I should have brought a mic but never mind, and meanwhile do you have a mic handy, thank you. All right, so he says cooperate but unfortunately he's been matched against the nasty uncooperative ant. It's not that he's nasty he's just been hardwired to say?

Student: Not cooperate.

Professor Ben Polak: Not cooperate, to say defect. So what's going to happen now? So unfortunately for Nick he, the cooperative ant, was matched against a not very nice ant, he was playing C and Rahul was playing D, so Nick's payoff is 0, so we went from one Nick to being no Nick, he got wiped out. Rahul meanwhile, he got a payoff of what? What was Rahul's payoff? Rahul's payoff is 3, so suddenly there's not just Rahul but there's, the mutation is growing. Now, we go on matching, each period some of you are going to be matched against other cooperative ants and you're going to be doing fine, but every now and then, actually increasingly often, you're going to be matched against one of our mutants and those mutants are going to grow, those mutants are going to go on growing. Now it's true that every now and then a mutant's going to meet another mutant, is that right? But we don't have to worry about that, all we have to worry about is, are they going to die out early on? It's going to be very rare for them to meet another mutant, at least early on.

So the strategy cooperate is evolutionarily stable if this small mutation, here's our small mutation, disappears and doesn't turn into a bigger mutation. Of course in this case, as we just saw, the mutation didn't die out, it actually grew. So to be formal about this, let me be careful, for the strategy cooperate to be evolutionary stable it needs to be the case that the mutation actually dies out on average, and here far from dying out, it grew. We now have lots of these nasty T.A.'s. People understand? All right, so let's just, thanks. So let's just try and do that a little bit more formally on the board. That was the basic idea, and the graphics of the idea, but let's look at it a little bit more formally. So we basically have cooperative ants and a few mutant defecter ants out there, so mutant not-cooperative ants, and what we want to keep track of, are their average payoffs in random matches.

So the incumbent ant, the cooperative ants, they're playing, we're interested in their payoff and they're playing a population that's mixed. Almost everybody in the population is cooperative, so let's say 1 - ε of the people they're playing against (where ε is a small number, a very small number) are also cooperative, but every now and then, like our poor friend Nick, they're going to come across a nasty mutant, they're going to come across Rahul. So what's their payoff going to be on average, their average payoff? So 1 - ε of the time, we can actually think of this mix here, so it's 1 - ε and ε. 1 - ε of the time they're going to meet another cooperative ant and get a payoff of 2, but ε of the time they're going to meet Rahul and get a payoff of 0. Is that correct? So that's their average payoff. Now how about the mutants' payoffs? So the mutants, there aren't many of them around, but they are also playing against the same mixed population, 1 - ε of the time they're going to be matched against a cooperative ant and ε of the time the two mutants, the two T.A.'s are going to playoff against each other.

Let's have a look at what their payoff is. So their payoff is 1 - ε, they get a payoff of 3, that's what we saw when Rahul met Nick and the other ε of the time, Rahul meets Jake or somebody and has to make do with a payoff of 1. But if we keep track of this, what do we have? This equals 2(1--ε) and this equals 3(1--ε) + ε. Is that right? Clearly, I hope this is clear, the payoff to the mutant is bigger. Everyone happy with that? The payoff to the mutant is bigger which means the mutation is not going to die out. In fact it's going to grow. Mutation's not going to die out. So we can conclude that C is not evolutionarily stable. I'm going to start using ES for evolutionarily stable. So in this particular game cooperation is not ES. So we might ask what is ES in this game? Well that's not going to be hard to figure it out, since there is only one choice. I claim that D is going to be ES, but let's just prove it.

So is it going to be the case that these mutants will eventually take over the whole population and everyone will end up looking like a T.A. and being uncooperative? So they're busily asexually reproducing all the time, they get bigger and bigger, and isn't it in fact the case that once they've conquered everything that they're ES? So is defect ES in this game? To figure that out we have to do exactly the reverse experiment. So the experiment now is imagine everyone in the population, all of you guys, all of the students in the room are nasty non-cooperative defecting ants. You're going along and most of the time you're meeting, your meeting each other all the time for now and you're getting a payoff of 1. And now there's a mutation, but this time the mutation isn't the scary non-cooperative mutation that we saw before with Rahul, it's going to be Myrto over there. And Myrto is a nice mutation who cooperates. So Myrto's our nice, cooperative mutation. Let me just get the camera to pan on poor little Myrto. So there is poor little Myrto who is the only cooperative mutation in the room. Wave, there we go okay, right.

What's going to happen to our cooperative mutation? So most of the time you non-cooperative people are matching up against each other. We're going to figure out what your payoff is. So that the uncooperative incumbents, they're playing a population that is 1 - ε non-cooperative and ε of the time they meet Myrto. So what's their average payoffs? Let's be careful, let's switch things around up here just to make sure we can see what happened. So switching things around on our chart up here, we're now looking at the case where there's 1 - ε non-cooperators and ε cooperators. So you guys, you non-cooperative ants are most of the time meeting each other, and when you meet each other you're getting a payoff of 1. So 1 - ε of the time you're getting a payoff of 1, but ε of the time you're doing great because you're meeting Myrto, and unfortunately you're beating up on Myrto.

How is she doing? Well she's a cooperator and 1 - ε of the time she's meeting you guys and ε of the time she meets another nice cooperative ant like Jake. So her payoff is 1 - ε of the time she gets nothing and ε of the time she does pretty well and gets 2, but it's only ε of the time. Is that right? So far, so good. So what's going to happen to this mutation? Well the incumbent population, their average payoff comes down to being 1 - ε + 3ε and Myrto's payoff, the mutant ends up being 2ε. So unfortunately, these nasty ants are thriving and Myrto gets wiped out, so she can sit down again. So what have we shown here? We've shown that defect, not cooperate, is evolutionarily stable in this game. Any mutation, there's only one possible mutation, any mutation gets wiped out.

Chapter 2. Game Theory and Evolution: Evolutionarily Stable Strategies - Discussion [00:25:40]

Think about these two different models here.

The first, well not different models but different metaphors, so we started out with this population of relatively nice people and we had a nasty mutation, so think of the movie Alien, actually don't because it's a horrible movie, but all right and if you remember that horrible movie, Alien or Species, think of the movie Species, an even worse movie. Think of the movie Species. That was Rahul and he grew albeit asexually, very rapidly. That's a pretty scary movie, right? Conversely, when we had this population of nasty people and we tried to have a nice little invasion of a nice mutant, so think of the movie ET, unfortunately, ET got squished, that's the two extremes. So what is the lesson that we can actually draw from this?

So there's a lesson here and the lesson--I guess there's going to be two lessons here but let's start with one. The first lesson here is that,--well let's just put it on the board first of all. Nature, the outcome of evolution, nature, can suck. So again we could use a more formal term but this is an important lesson. There's a tendency for some people to think that if something arises as a consequence of evolution, if something is natural, if something is in nature, it must therefore be good in some moral way, efficient or something. What we're seeing here is in this game the consequence of nature is a horrible consequence. For those people who doubt that nature can be pretty unpleasant have a look at yesterday's science page of The New York Times, and read the piece about how the baboons basically kill the baby baboons. The biggest cause of death among baby baboons is infanticide. This is ES it turns out and they explain why in The Times. It isn't pleasant. So nature can suck, nature can be pretty inefficient.

Now, this raises a question in this particular game because look at the examples we started with. We started with lions who are thinking about--or not thinking about, sorry, they weren't thinking about--they were hardwired to go after antelope and either cooperate or not. And we talked about ants who are hardwired to defend the nest or not. We all know from watching endless nature shows as children on TV that actually lions do cooperate when they go after antelope, and actually ants do defend the nest when it's invaded by a spider or something. Is that right? Looking for some nodding, is that right? Yeah okay, good. So what happened? What's wrong with our model here? We've argued here that, in this model, nature's going to produce this non-cooperative behavior. We know there are examples of cooperation in nature, what's going on? What are we missing? Can I get a mike in here. It's here. Let me do it. I'll do it. Yeah, so what's going on?

Student: Different communities might compete against each other, so one community that doesn't have any of these mutations and stays cooperative might succeed, so communities in general don't have as many mutations.

Professor Ben Polak: Okay, so that's an interesting idea. So part of this is because we're focusing on within species competition rather than the cross-species competition. That turned out to be a complicated idea and large literature on it and it's actually a little difficult. I was looking for, but it's a good suggestion, I was looking for something simpler actually, let me come back here.

Student: Do these species have ways to detect cheaters and punish them?

Professor Ben Polak: Okay, that might be a possibility and certainly there is some evidence of that but not much. Again, we're looking at pretty high primates by the time you get to that. Something simpler, something you should all have on your mind all the time if you're normal teenagers.

Student: The payoffs could change as the proportion of cheaters increases, so you're not going to get the payoff of 3 if everyone's cheating.

Professor Ben Polak: All right, but okay, it's true that you're going to get a different expected payoff, but that isn't the payoff of the game. I'm looking for something really simple that, let me repeat, I'm guessing is on all of your minds as teenagers. Let's see if that hint's going to get the right answer. Yeah.

Student: Answer, not asexual.

Professor Ben Polak: Right, the assumption that's really driving this here is the assumption of asexual reproduction. Sexual reproduction with gene exchange is going to make a difference. Why? I don't want to delve too much into the biology here, but the main why is, what matters is the survival of the gene, not survival of the individual ant or individual lion. So if you have sexual reproduction you get gene redistribution among children and cousins to a less extent, so provided the other ants in the nest are closely enough genetically related to you, and provided the lions are close enough related to you, then it may turn out to be, it may turn out that you will cooperate, and that in fact, does change the payoffs of the game. It doesn't change the payoffs for the individual lion or ant, it changes the payoffs for the gene.

Chapter 3. Game Theory and Evolution: Evolutionarily Stable Strategies Are Always Nash Equilibria [00:30:42]

Now if we have time we'll come back and look at that on Monday, and even if we don't have time, in the reading I've left for you in the reading packet, it goes into that in some detail. It's a much more complicated model but that's what we're going to get, for the most part that's where we're going to get cooperation from rather than from the other points that people made. So sex can make a difference here, but we're going to stick with asexual reproduction because it's easier to analyze and stick with it for the minute. And we've learned, the second lesson we've learned here, I'm going to generalize from this lesson, we learned that a strategy that was strictly dominated was not evolutionarily stable. I've only shown an example here but we'll extend from the example.

So if a strategy is strictly dominated, and of course cooperate is strictly dominated here, then it is not evolutionarily stable, at least in this simple game of just asexual reproduction. Even though I haven't proved this here, the idea is exactly the idea of this example, so let's just try and talk it through. Suppose a strategy was strictly dominated, how do we know, how can we see that it will not be evolutionarily stable? Somebody? Where are my pre-med majors? Where are my pre-med people? They're all hiding now. I know you're out there because I've seen your forms, but never mind. Why is this dominated strategy not evolutionarily stable?

Student: The strategy that dominates it will be a successful mutation.

Professor Ben Polak: Good, so the strategy that dominates, the strategy that does the domination of this strictly dominated strategy would be a successful mutation. If it enters it does well, not just against this strategy but against any mix involving itself and this strategy. So the strictly dominator strategy will invade. So we can't have strictly dominated strategies surviving in evolution. Let's do another example and we'll see if we can learn some more. So this is going to be a slightly more complicated example. Let's have a three-by-three game, so here's our three-by-three game and we'll just label the strategies A, B, C and A, B, C and once again we're going to focus on symmetric games, so this game will be symmetric: 2, 2, 0, 0, 0, 0, 0, 0, lots of zeros in this, 1, 1, 0,0, 1,1, 0,0. So this is a symmetric game. We'll look at this game a little bit, I don't want to look at all of it, I want to ask the question is strategy C evolutionarily stable?

Let's go back to our experiment. What does it involve? Imagine everyone in the room is hardwired to play C, could that population be invaded by a mutation? So what do we think? Is C evolutionarily stable? Who thinks it is stable? Who thinks it's not stable? So most of you think it's not stable so you must have some idea why. Why do we think it's not stable? Let me just try and cold call a little bit. Anybody? Let me sample somebody. If it's not stable it must be it's going to get invaded, so who's going to invade it? Can we try this gentleman here, have a guess who's going to invade it?

Student: Strategy B.

Professor Ben Polak: Strategy B, okay that's correct. Your name is?

Student: Greg.

Professor Ben Polak: So Greg is saying strategy B might invade here, let's have a look, let's see what happens. So suppose there's an invasion of B, so again the T.A.'s are playing B, everyone else is playing C, and let's see how these incumbent genes do. So C is playing against 1 - ε of the population who are playing C and ε who are playing B and its average payoff? Well 1 - ε of the time it'll be matched against essentially itself and get a payoff of 0, but ε of the time it will be matched against a B and get a payoff of 1. Everyone agree with that? So 1 - ε of the time it meets itself and gets nothing, ε of the time it gets lucky and it meets a T.A. and gets a payoff of 1.

How about those B invaders, the T.A.'s in the class. So 1 - ε of the time they're going to meet C's and ε of the time they're going to meet B's and 1 - ε of the time therefore, the B is meeting a C and getting a payoff of 1, so 1 - ε of the time the T.A. is meeting one of you. And ε of the time the T.A. is meeting another T.A., playing B against B, and getting 0. So this works out as ε and this works out as 1 - ε. But notice that if ε is small, if there were just a small number of these mutations, 1 - ε is bigger than ε. Is that right? So that means that this mutation will not die out, it'll probably go on, we can actually say a bit more, it'll probably go on growing until it's roughly half the population. But, in particular, it won't die out. Since it won't die out, since the mutation doesn't die out, we can conclude that C is not evolutionarily stable. Everyone okay with that idea?

Now this idea, this example is a little bit more complicated than the previous example, because it turns out that the invading mutant population, the population B is itself not evolutionarily stable. Everyone see that? C wasn't evolutionarily stable because it got invaded by B. Who's going to invade B? C is going to invade B, turns out everything is exactly symmetric here. So even though we're arguing that C is not evolutionarily stable because it's invaded by B, it doesn't have to be the case that the successful mutant is itself evolutionarily stable. So in this particular example the invader, namely B, is itself not evolutionarily stable. Nevertheless, it doesn't die out but it invades the population of C's. But there's a second observation I want to draw off this one, hoping there's a bit of chalk here, not finding it, there's a second observation I want to draw here, what do we observe?

What else is true about both C and B? So what's true about everyone playing C or everyone playing B? What else? Just going back to what we've learned in the course so far, what do we think about everyone playing C? So if we had everyone playing C we'd be looking at an object like this (C,C) and we might ask the question is (C,C) what? Is it a Nash Equilibrium? Is (C,C) a Nash Equilibrium? Here we asked was it evolutionarily stable, we found it wasn't. Let's ask a different question, and forget evolution for a second. Is (C,C) a Nash Equilibrium? Well that's a question you all should be able to answer for the mid-term so somebody tell me the answer. It's not a Nash Equilibrium, how do we know it's not a Nash Equilibrium? What do we have to do to show it's not a Nash Equilibrium? We have to show that there's a profitable deviation, right. So what's the, shout it out, what's the profitable deviation here? Profitable deviation is B. It's not no because B is a strictly profitable deviation.

Notice that the thing that was a strictly profitable deviation was the same thing that would have invaded the population of C's in the context of our ant's nest and the context of this classroom being an ant's nest and looking at evolution. Is that right? So what have we just learned here? Well this idea turns out to be general. The idea is if a strategy S, perhaps I should make this a lesson, if a strategy S is not Nash, so in other words (S,S) is not a Nash Equilibrium, then S is not evolutionarily stable. If S is not Nash, then S is not evolutionarily stable. Now with a little bit of logic, can we flip that around and we will say, what that actually tells us is, it's equivalent to saying, if S is evolutionarily stable then (S,S) is a Nash Equilibrium, that's the equivalent way of saying that. So what's the idea here?

Let's look at the first line rather than the second line, because it's pretty easy to understand. The idea here is, if a strategy is not Nash, what that means is, is there's some other strategy that would be a strictly profitable deviation and that's enough to tell us that S cannot be evolutionarily stable. Why? Because take that same strategy that was a strictly profitable deviation, B in this case, and that strategy can be thought of as the mutation that's going to invade the strategy we started from. Everyone see that? Say it once more. Suppose a strategy isn't, suppose strategy (S,S) is not a Nash Equilibrium. What that means is some other strategy S' say that is a strictly profitable deviation. Now think about strictly profitable deviation S' as a mutant invasion, so now Rahul is playing, is hardwired to play S'. Since it was strictly profitable it's going to be, since it was strictly profitable as a deviation, it's going to be successful as an invader. So again I haven't formally proved it, but that's exactly how the proof would run.

Chapter 4. Game Theory and Evolution: Nash Equilibria Are Not Always Evolutionarily Stable Strategies [00:42:32]

So let's pause for a second and just see where we are. What we've managed to show is there's a connection here between dominance and evolutionarily stable. We've said if a strategy is not dominated it cannot be evolutionarily stable. We've also begun to show a connection between Nash Equilibrium and evolutionary stability. So two of the ideas we've developed over the last six weeks are re-emerging in this completely other context of animal behavior. The idea of dominance and the idea of Nash, and so far how far have we got? We've said that if something's going to be evolutionarily stable it better be the case that it's also Nash. That raises I think a natural question which is, is the opposite true? It would be really pretty great if it were true the other way around. If I could show that if a strategy was Nash then it would necessarily also be ES. That would be pretty cool, right? This idea we've spent a lot of time developing in the class turns out to be the key idea.

Unfortunately, life isn't quite so neat. Let's see why. So what I'm going to do is, I'm going to show you an example and what our example's going to illustrate is we can find Nash strategies that are not evolutionarily stable, and the example is embarrassingly simple. Here's a game, it's a two player, two strategy game and the payoffs are (1,1), (0, 0), (0, 0), (0, 0). This is kind of an embarrassingly simple game. What are the Nash Equilibria in this game? Well let's go through slowly. If Player row is playing A, then column's best response is to pick A and if player row is playing B, then column's best response is either to play A or B. If column is playing A, then row's best response is to choose A. If column is playing B, then row's best response is either A or B. So this is the kind of exercise that is almost second nature for you guys now. So we can conclude by looking where the Nash, where these best responses coincide, that the Nash Equilibria here are (A,A) and (B,B).

Everyone happy with that? You should be looking like yeah this is easy, right? So all right, but let's look at this second Nash Equilibrium. The (B,B) Nash Equilibrium, the one that's down here. Is B evolutionarily stable? Well again let's think about it, let's just think through the exercise. Suppose the entire class were playing B. Wake up the guy in the middle and tell him he's playing B too. The whole class is playing B and suppose there's an invasion. What could the invasion be? The invasion better be an invasion of A's. What's going to be the expected payoff or the average payoff of the incumbents, of all of you in the class who are playing B? Well, without doing it too laboriously, 1 - ε of the time you're going to meet another student in which case your payoff, in which case you'll be playing B against B and your payoff will 0 and ε of the time you're going to meet a T.A. and these T.A.'s are playing A, but again, you'll get 0, so your average payoff will be what? 0, your payoff will be 0.

So if you're an incumbent and you, against this mix, your payoff will be 0. And if you're an invader? So how is Rahul doing this time? Rahul's playing A. 1 - ε of the time this A is playing against B, so 1 - ε of the time he's playing against a B. So 1 - ε of the time Rahul meets a student and gets a payoff of 0. But ε of the time Rahul hits it lucky and Rahul meets another T.A., Rahul meets Jake and when he meets Jake his payoff is 1. Is that correct? So his total average payoff is ε, which is bigger than 0, so indeed, it turns out that Rahul's gene is going to grow, the B's are going to shrink. (B,B) was Nash but it's not evolutionarily stable, it can be invaded. Everyone see that? Everyone see how that invasion worked? So the key to that invasion was when Rahul met another incumbent student he did no better than the students did against students. But on those rare occasions when Rahul met another T.A. he made hay--or not hay, it's asexual reproduction--he got a payoff of 1. Those rare occasions were enough to make Rahul grow and thrive, whereas the B's relatively speaking, shrank. Everyone happy with that?

Okay, so what we have here is an example of something that is Nash but is not evolutionarily stable. Can anyone say what's special about this example? How did I rig this example? There's something really kind of knife edged and rigged about this example. What's rigged about this example? Anybody? I don't want to cold call on this. No takers, everyone's kind of in that pre-midterm scared mode. This guy is rescuing, this guy is saving the class today. Someone else has to get in. What's your name again, I forgot your name?

Student: Steven.

Professor Ben Polak: So Steven is saving the rest of you, but go ahead Steven.

Student: B is only a weak best response to anything.

Professor Ben Polak: Good, so what's true about this example is that although B is a best response against B, it only is so weakly. Is that right? It's only so weakly. If, in fact, we got rid of Nash Equilibria that relied only on weak best responses, then we wouldn't be able to produce this example. So, in particular, if we looked at Nash Equilibria, where the Nash strategy was strictly a best response, it was strictly better than playing any other pure strategy, those cases would be evolutionarily stable. Anybody remember what we call Nash Equilibria where the Nash strategy is a strict best response? We call them strict Nash, so it is in fact true, is if (S,S) is a strict Nash Equilibrium, by which I mean S is a strict best response to S, then S is a Nash equilibrium--sorry, S is evolutionarily stable.

Okay, so we've been pretty, we've been only semi-formal so far. I haven't really put up a formal definition, and what I've been groping towards slowly is a connection between an idea growing out of biology, namely evolutionarily stable, and an idea growing out of Economics and Mathematics, namely Nash Equilibrium. What I want to do now is I wanted to show you formally how are these connected. I'm not going to prove this part of the class, because a proof is a bit lengthy, but I will include a proof on the handout that's going to be available this afternoon. I'm just going to argue it for now. Here we go, there's some chalk. So what I want to do first is I want to write down a formal definition of evolutionary stability. It's a definition that comes out of biology and it's a little bit of a, you're going to see it's a little bit of a mouthful this definition. So for those people who are a little bit sleepy, you need to wake up a little bit, because it's going to be a little bit of a difficult definition. But this is the definition that came out of biology.

This is a formal definition. It comes out of biology, and in particular, it's due to a guy called Maynard Smith who wrote this in 1972. So obviously the earlier idea is due to Darwin, but this formal idea is due to Maynard Smith. So here's our formal definition. So in a symmetric two player game--so everything we've been looking at so far --, the pure strategy S--let's call it Ŝ,--let's give it a name --, is evolutionarily stable --and again I'll use ES for that, but let me be a little bit nerdy here. I'm going to say in pure strategies: it's evolutionarily stable in pure strategies--and I'm putting this in because we're going to come back on Monday and consider mixed strategies. So it's evolutionarily stable in pure strategies if, and just leave a little bit of a space here, because I'm going to need a little bit of space between these next two lines, leave a little space here.

So here's our big mouthful, what we need is 1 - ε of the payoff of playing Ŝ against Ŝ plus ε of the payoff of playing Ŝ against S' has to be strictly bigger than 1 - ε of the payoff of playing S' against Ŝ plus ε of the payoff of playing S' against itself. This has to be true for all possible deviations S', but we also, this is why we need the extra line, and for all mutation sizes ε less than some ε-bar. This is where I need to go back and just be a bit more careful, so I'm going to write on this extra line, there exists an ε-bar. That's why I left the extra line there. So this looks like very nerdy things. Let's talk our way through it. What's it saying? It's saying that Ŝ is evolutionarily stable if against all possible mutations, so all possible versions of Rahul S', the payoff of Ŝ against the subsequently mixed population is bigger than the payoff of S' against the subsequently mixed population.

Let's just see why that is so. So this on the left is the payoff of Ŝ against a population in which 1 - ε of the population like it is playing Ŝ and ε of the population is like Rahul, because 1 - ε of the time it meets something like itself, and ε of the time it meets a T.A. Conversely, on the right hand side, we have the payoff to the mutation. The payoff to the mutation is 1 - ε of the time the T.A. meets a student and gets the payoff of S' against Ŝ, ε of the time the T.A. meets itself and gets a payoff of S' against S', and we have this inequality we had before, that says the mutation invader has to do worse so it dies out. So the nasty bit of this definition, the part that's a little bit--it takes a while to get your head around--is this qualifier about the size of mutations. And all it's saying, ignore the math of it, all it's saying is this better be true for all small mutations. You could think of that saying, this has to be true for all small mutations.

So this definition from biology is exactly, it exactly mimics the argument which we went through several times now, both using the class and using the board and figuring out average payoffs. This is an ugly definition. Everyone agree this is kind of ugly? So now what I'm going to do is I'm going to give you an entirely different definition. It's also going to be ugly but a little bit less ugly. So think of that as Definition 1, and it came from biology, it came from this paper in 1972. I think it was in Nature by Maynard Smith and co-authors. Think of this other definition, that's coming out of Economics. So Definition 2, a strategy Ŝ--I should have had the same qualifier, in a symmetric two player game--same thing to start off with--a strategy Ŝ is ES in pure strategies--so same as we had before if two things. Thing number one, let's call it A, if (Ŝ, Ŝ) is a Nash Equilibrium--is a symmetric obviously--is a symmetric Nash Equilibrium of the game. We're not done yet. Let me just write down what it means to be a symmetric Nash Equilibrium of the game, what does that mean? That means i.e., the payoff of Ŝ against Ŝ must be at least as big as the payoff of S' against Ŝ for all S'. That's just a standard thing we've seen it many, many times, it says Ŝ is a best response against Ŝ.

We're not quite done yet. And B: what else do we need? We need, if this weak inequality I wrote above is actually an equality, if the payoff of Ŝ against itself is actually equal to the payoff of S' against Ŝ, then the payoff of Ŝ against S' must be bigger than the payoff of S' against itself. So this is still a bit of a mouthful, but I claim it's going to end up being--it ends up being a little simpler to think about. So let's see what it says again, it says Ŝ is evolutionarily stable if it's a Nash Equilibrium, that's basically this and if it's only a weak Nash Equilibrium--there's a tie --, then it better beat up on the mutant. It better beat up on the mutant when it meets the mutant. I'm going to try and give you an intuition as to why this is true in a minute, but first I want to tell you why you should care about this. So without getting too religious, I want to tell you why I think this is an exciting result. Why am I dragging you through more algebra than usual?

So I'm going to give you two reasons why this is an important result. The first is, it's going to turn out that when we analyze games it's very easy to check Definition 2. It's going to turn out it's very easy to check Definition 2. It's really rather a pain to check Definition 1. Why? Because you got to keep track of these ε 's and so on, but the fact that Definition 2 is equivalent to Definition 1--did I say that? That's the point. The fact that Definition 1 and 2 are equivalent--so 1 is equivalent to 2--the fact that these definitions are equivalent means we only have to check the second definition and that's easy. So when biologists are setting up experiments involving wasps or ants, or lions, or chimpanzees, provided they can check this in the game they're done, and that turns out to be easy. And we'll see that on Monday. That's the sort of instrumental reason.

Now I want to give you the "religious" reason why you should care about this. So let me just try--and I often find that this appeals to like a third of the students and the other two-thirds of the students think I'm completely bonkers at this point. So that's fine, you can think I'm bonkers, that's fine. For the third of you who are nerdy like me, I want you to just see the appeal of this. Here we have two ideas. One idea arises out of biology. I say it rose from Maynard Smith but it really comes from Darwin. So it's a nineteenth century idea in biology. It's probably the most important idea in biology in the nineteenth century. It may be the most important idea in biology for 200 years. Is that a fair judgment? The notion of evolutionary stability. The other idea comes out of Economics. It's looking at Nash Equilibria and strict Nash Equilibria and so on. Up there we've got biology; down here we have Economics. This is an idea that emerged in Economics in the twentieth century in the 1950s, so roughly 80 years after Darwin. This is a big idea in Economics, this is a big idea in biology. So what appeals to me because I'm kind of a nerdy kind of guy is I think it's kind of wonderful that those two ideas are almost the same. I think it's a beautiful thing that those two ideas are kind of the same thing.

But now you are looking at me and thinking I'm bonkers. So let me push it harder. So think back to the, think about great intellectual coincidences of earlier periods. Think about the seventeenth century. In the seventeenth century people figured out that the mechanical laws that governed the rotation of the planets were at least approximately the same as those that governed a clock. The basic laws of Newtonian Physics were the same, and they thought this was a wonderful thing. They have these completely different areas of intellectual pursuit and they turned out to be the same. In the scientific revolution, it turned out they were the same. For those people who chose to continue believing in God at that point, they thought of God now as being what? As being a heavenly clock maker. It was a wonderful moment.

So here, in our own time, or perhaps just before it, we see a similar thing. Here we see one of the most important ideas in science coinciding with one of the most important ideas in Economics. Where does this lead us? Well now I guess, I mean the bad news is this may make a lot of people doubt the existence of God I guess. [That's going to get me in trouble, I didn't say that.] But at least if you did go on believing in God, you're going to have to believe God's an economist, that's a pretty good thing.

Chapter 5. Game Theory and Evolution: Evolutionarily Stable Strategies and Nash Equilibria - Discussion [01:03:00]

So that's my little religious piece on this. All right, now I want to spend the last ten minutes, having told you this wonderful thing, trying to convince you that it's true, and again, there's a proof on the handout that you can read on the web. I just want to give you an idea of why it's true. Okay, so here we go. What I want to convince you of is that this definition, this kind of Game Theory definition, this Econ definition implies the one above. The one above we kind of already argued several times today, corresponds to the notion of evolutionarily stability. We talked about it several times. We wrote down these kind of equations several times. So I want to convince you that this one implies that one. So what do I have to convince you? Let's imagine, let's fix, there's no way of getting you to see that one without--no, never mind, it was a bad idea. The embarrassment of playing with heavy boards here. Alright let's try and shift this one up. If you ever think of asking your dean for some new technology for Yale, we might think about having some slightly lighter blackboards. Just for the evolutionary stability of the average weakling economist.

So what I'm going to do is let's fix a strategy Ŝ and suppose (Ŝ,Ŝ) is in fact Nash. What I want to convince you of is that Ŝ is going to be evolutionarily stable. So I claim there's two possibilities, there are two cases. The two cases are either it's the case that Ŝ against Ŝ does strictly better than Ŝ against S' for all S'. So let's just be careful, since we know its Nash, we already know that the payoff of Ŝ against itself is at least weakly better than any possible deviation, that's what it means to be Nash. So everyone agree with that? We know that Ŝ is a best response to Ŝ, so it must be weakly better than any possible deviation. So let's take the first case, where actually it's strictly better, and let's go back to our classroom example and suppose that--let's go back to the first definition by going back to our metaphor of the class. So you guys are all ants, and suppose that every student in the class is playing Ŝ. You're all playing Ŝ and suppose it's the case that in fact the payoff of Ŝ against Ŝ is bigger than the payoff of Ŝ against S', and suppose that a mutation arises, sorry Rahul here we go again. So as the mutation arises, here's Rahul our mutation, and he's going to be randomly matched against one of you guys. So let's compare Rahul's payoff against this gentleman's payoff. Stand up a second. Your name is?

Student: Pat.

Professor Ben Polak: Pat, all right. So Pat is our typical Ŝ incumbent ant, doesn't look like an ant but never mind, stretch your imagination a bit. Most of the time what's happening? Most of the time Pat is being matched against one of the rest of you, and when he's matched against one of the rest of you, he's getting the payoff Ŝ against Ŝ, which is U(Ŝ,Ŝ), that's what he's getting most of the time. Every now and then he's meeting a mutant. Okay fine, so every now and then he's getting a slightly different payoff, but most of the time he's getting a payoff which is U(Ŝ,Ŝ). What about Rahul? So Rahul, okay every now and then he's going to be lucky and meet another T.A., but most of the time, almost all the time he's being matched against one of you, for example he's matched against Pat. When he's matched against Pat, his payoff is U(Ŝ,S') which is lower. That right? So Pat's payoff almost all the time is U(Ŝ,Ŝ) and Rahul's payoff almost all the time is U(Ŝ,S'). But our assumption is U(Ŝ,S') is lower, so Rahul's going to die out. So in this case, Rahul dies out because most of the time he's playing an incumbent and just doing horribly.

Everyone convinced by that? So in this case, the mutant dies out. So the mutant dies out because she or he meets Ŝ often. The mutant meets the incumbent often and just does horribly against the incumbent. That's the first case, that's case one and case two is, it could be the case that in fact Rahul does pretty well against the incumbents, but unfortunately for Rahul, it could be the case, sorry, but in that case according to Definition B, Rahul doesn't do so well against other T.A.'s. So this is case two, case one was Rahul just gets beaten up whenever he meets one of you, sorry Rahul you are getting beaten up a lot here, but Rahul gets beaten up when he meets one of you, and here's the case where Rahul is doing fine against you, but he does horribly against incumbents, so that's the harder case. Let's talk through that case. So back up again, sorry Rahul you have to get up again, so once again you guys are all playing Ŝ's, there's a mutation of Rahul's playing S', and let's get Pat up again. So when Pat meets one of you guys, so let's pick out one of you guys, so let me pick out, sorry your name is?

Student: Christine.

Professor Ben Polak: Christine. So when Pat meets Christine, which is most of the time, he does okay and when Rahul meets Christine which happens most of the time, he does exactly the same. So if you just compared them in their random matches against Christine all the rest of you, they're the same. But every now and then instead, every ε of the time, rarely, Rahul is going to meet Jake, Jake's writing, he's going to meet Kaj, stand up a second and on those occasions let's see what happens. So when Pat meets Kaj he does pretty well, and he gets some big bonanza, he gets a much bigger payoff than when Rahul meets Kaj, they just do horribly against each other. The fact that they do horribly against each other causes them to die out. So let's just talk this through. There were two ways in which Rahul could die out. One is he dies out because he does horribly against you and the second way he can die out is if he does equally well against you as you do against yourself, but he does horribly when he meets the other T.A.'s. In either case, you guys are evolutionarily stable and Rahul is in trouble.

Was that convincing enough an argument? There's a formal proof in the handout but this will be enough to get us started. So in this case, the mutant does okay against Ŝ but gets clobbered, does badly let's say against S'. So mutants that are going to die out are those that do badly against incumbents or do badly against themselves. In either case Ŝ will be evolutionarily stable. Now next time I want to take this further in two directions. I want to see what happens about ES in more complicated games, for example, in cooperation games and we're going to see that evolution doesn't do great in cooperation games. We'll also, if we have time, going to look at sexual reproduction, which is probably what you guys are all interested in anyway. I'll see you all on Monday.

[end of transcript]