Game Theory

ECON 159 - Lecture 13 - Sequential Games: Moral Hazard, Incentives, and Hungry Lions

Chapter 1. Sequential Games: Backward Induction [00:00:00]

Professor Ben Polak: Okay, so I want to set up a new topic today, and to get us started I thought we’d play a game. So the game is going to be called cash in a hat, and we’ll see what this game is like in a minute. So the idea is this, there’s going to be two players. I’m going to pick on two of you. I hope I can find people who brought some money to class. So Player I will have a choice, Player I can put nothing, $1 or $3 into a hat. The hat will then be given to Player II and Player II can look at what’s in the hat and either she can match what’s in the hat, i.e., add the same amount in so 1 if it’s 1, or 3 if it’s 3, or she can simply take the cash.

The payoffs of this game will be as follows. Player I, if they put nothing in, they’re not going to get anything. If they put 1 in and it’s matched then they double their money, so they get 2 back. If they put 3 in and it’s matched then they double their money, so they get 6 back so they’ll net 3. However, if they put 1 in and Player II takes it they’ve just lost their dollar, and if they put $3 in and Player II takes it they’ve lost $3. Everyone understand that? Pretty simple. From Player II’s point of view, if Player II matches then they get their investment back plus $1.50 if they matched 1, so the get 2.50 back in all; and they get their investment back plus $2 if they match 3, so they get 5 back in all. Of course, if they simply take the money out of the hat and put it in their pocket then that’s what they get in the game.

So once again, if Player II matches with $1, there’s $1 in the hat and she matches, then she gets back 2.50 for a profit of 1.50. If there’s $3 and she matches 3, she gets back 5 for a profit of 2; and if she simply takes the money out of the hat and puts it in her pocket then she gets how ever much money was in the hat. So I here will be the provider of the hat, we’ll use the same hat we used before, and if necessary I’ll provide sheets of paper, we can write IOU’s on, but I’m hoping we can play for real cash. Later on–it would be nice to get a chair here–but for now, let me just go down here and I’m going to grab this mike. Who here wants to be Player I? Who brought some money to class today? I’m quite willing to pick on somebody. All right good, so Player I is, what’s your name again?

Student: Justin.

Professor Ben Polak: Justin. So hold onto that hat for a second and who wants to be Player II? All right, so this gentleman here whose name is?

Student: Nate.

Professor Ben Polak: Nate. All right so we have Justin and Nate. So Justin you have to decide how much money you’re going to put in the hat. Did you bring some money too Nate? Better make sure you have, otherwise borrow some from your neighbors.

Student: I’m going to put in a buck.

Professor Ben Polak: All right, so the money’s being put in the hat and delivered to Nate.

Student: I’m going to put in a buck too so that I get 2.50.

Professor Ben Polak: All right, so if we empty out the hat, here’s our beautiful pink hat. I should really have the hat passed across the room rather than have me do it, but we have $2 in the hat so I’m going to pay each - I’m going to pay–what am I going to pay? I’m going to pay Player I $2, I’m going to give them back their original $1 and give them $2 in a second, and I’m going to pay Player II $1.50. Just hold the hat.

Student: $2.50.

Professor Ben Polak: $2.50, you got your money back. Trying to cheat me is that it? So Player, I’ll get you the rest in a second, hang on a second for now, so we double the money. So one reason to take this class is, if you play well, you get money. All right, now let’s just play again, so we had one round of this, let’s have another round. Who wants to be Player I now? All right, so Katie is Player I. Katie did you bring some money?

Student: Yeah, sure.

Professor Ben Polak: All right, and who wants to be Player II? Someone further away, let’s have, Steven isn’t it? Is that right.

Student: Yeah.

Professor Ben Polak: So Steven’s going to be Player II. Did you bring some cash?

Student: I think so.

Professor Ben Polak: All right, otherwise borrow it from your neighbors, I have IOU’s here. So here’s the hat. You’ve got coins great, so pass the hat down. Let’s see if it makes it there, test the honesty of the class here. It’s working its way down, I’m going to have to, there we go, all the way down to Steven. Check how much is in there.

Student: There’s a dollar in here.

Professor Ben Polak: There’s a dollar there so you can decide whether to match or not.

Student: Okay, I’ll match it.

Professor Ben Polak: All right, so Steven’s going to match as well. Once again, we have this time, rather annoyingly $1 and some coins in here, which I will empty on the stage to prove to everybody and I will–all right there are coins in here–so I’ll match this in a second and give it back to you guys. So everyone understand the game? Everyone understand how this game works? So we’re going to spend a while discussing this. We’d go on playing except I’d eventually lose so much money I’m going to run out of lunch money and the T.A.’s would object. All right, so what I want to do is I want to analyze this game. I want to think about what this game’s about, what’s really going on here.

So first this is just a little game involving putting some money in a hat. What I want to suggest to you, that this is a toy version of a much more important game. This is a toy version of a game involving a lender and a borrower, so you could think of our Player I’s–that was Katie on the second round and is it Justin on the first round, Justin on the first round. Imagine them working in an investment bank, or perhaps in a venture capitalist firm, and what they’re doing is they’re giving a loan to somebody, some budding entrepreneur who’s come up with a new project. So Steven in Round 2 and who was my budding entrepreneur in Round 1? What was your name?

Student: Nate.

Professor Ben Polak: Nate in Round 1 have come up with some budding project. They’ve left Yale, perhaps after their junior year. They’ve left early because they’ve got some great idea which is going to make them millions. So either it’s a new mouse trap or a new version of Facebook or something, and they go to the–and that’s Harvard isn’t it, so something better than a new version of Facebook–and they go to this venture capitalist firm. Maybe it’s some firm in New York or maybe it’s the Yale Investment Trust or whatever, and they explain this great idea to them and they ask for some money to invest in this firm to buy machinery and to pay early wages and so on.

The lender, the guy who works at the venture capitalist firm, has to decide how much money to invest in this project. After this money has been invested in the project, the person who borrowed it faces choices. They could go forward with their project, work hard, spend the money on the things they’re supposed to spend, or they could just disappear to Mexico (or go back to Yale for that matter). There’s actually something in between they could do which is less dramatic than disappearing to Mexico, they could just not work particularly hard, shirk, perhaps join in someone else’s project and just let the money run down without the lender really knowing about it. So this is a very real problem that some of you, some of the budding entrepreneur’s in the room are going to face when you leave Yale, and more of you, who are going to end up working for investment bank’s, perhaps unfortunately, are going to face as well.

Okay so we’re going to spend the whole of today analyzing this, but before we even start let’s notice that there’s something different about this game, other than its sheer triviality. There’s something different about this game than games we’ve played all the way up to the mid-term so far. What’s different about this game? Can I get a mike in here? Yes what’s different?

Student: Moves aren’t simultaneous.

Professor Ben Polak: The moves are not simultaneous. This is a sequential move game. Put the rules up here and bring this down, so for the first time we’re looking at a sequential move game and we’re going to spend most of the rest of the term looking at sequential move games, or at least looking at games that involve sequential elements. Now what’s really, what makes this a sequential move game? Let’s be careful about this. What makes it a sequential move game is not simply that Player I moved first and Player II moved second, although that’s true, what really made this a sequential move game is that Player II knew what Player I had done before he got to make his move. It isn’t the timing per se that matters, it’s going to turn out it’s the fact that Player II could observe Player I’s choice before having to make his or her choice that matters.

Notice, just while we’re on the subject, more than that Player I knew this was going to be the case. So let’s just get that down. So in this game Player II knows Player I’s choice before she chooses–she being Player II, let’s say before II chooses. Second, Player I knows that this will be the case, and the way we dramatized this just now is Player I put the money in the hat, the hat was transported across the classroom with people very honestly not stealing the money to Player II, Player II could look in the hat, could see how much money was in there, before Steven or Nate had to make that choice. Is that right?

So how do we go about analyzing games that have this sequential structure? So it turns out that a useful way to analyze them, and the way I want to get us used to today already, is to draw a tree. So in the first half of the class we drew a lot of matrices, now we’re going to draw trees. For those people who haven’t seen trees before, not a big deal. It’s an object that looks like a tree, that’s why it’s called a tree. So let’s have a look, so in this particular game, here is Player I making her choice and in fact she has three possible choices. She can put nothing, 1 or 3 into the hat. Player II then chooses, although really Player II only has choices to make if there’s some money in the hat, and Player II has two choices in this particular game in either case.

Either Player II is going to add $1 or $3 into the hat or they’re going to disappear off to Mexico, taking our investors’ lunch money with them. Let’s put down the payoffs. So let’s put down as payoffs, the net gains they make. So if nothing goes in, no one makes any money so (0, 0) will be the payoff. If 1 is matched by 1 then the investor, the lender doubles her money so she makes a net profit of 1. If the money is taken out by Player II then she loses 1. Down here if II matches her investment of 3 she doubles her money again, so she makes a net profit of 3, and if the money is taken out she loses 3. Everyone all right with that? Let’s do Player II’s payoffs. In the case in which he matches 1 he makes a net profit of 1.50. In the case in which he just steals the 1 he makes a net profit of 1. And down here, in the case where he matches 3 he makes a net profit of 2–he gets 5 back but he’d invested 3 to start with –, and here if he just takes the money out he makes a profit of 3.

So this game is just illustrated on the board, and for some of you this is the first time you’ve seen a tree, but I’m going to claim it isn’t such a complicated object and we could spend a lot of time analyzing this in terms of connected graphs and so on, but it isn’t really worth it: it’s just a tree. Okay, so let’s just make sure we understand that the payoffs here represent Player I’s payoff and Player II’s payoff respectively. So how do we analyze what to do in these games? It’s a good idea always to write out the tree like it’s a good idea to write out the matrix, but how to analyze what we should do? Well before we do that, let me just grab the mike and find out from our players what they should do. So where was my first investor? So Justin why did you put $1 in there?

Student: Well if I put $1 in then he has an incentive to put $1 in too. If I put in $3 he’s just going to take the money and run.

Professor Ben Polak: Okay, so what’s Justin doing? Justin is looking forward, if you like. He’s putting himself in Player II’s shoes. Who was our receiver? Nate. Nate’s shoes and anticipating what Nate’s going to do. And, as Justin said, if he puts in 3 Nate’s going to have an incentive to take the money and run. In fact, what would Nate have done? If 3 had been in the hat what would you have done?

Student: I like Justin but I would have taken the money.

Professor Ben Polak: All right, even though you know Justin, you would have taken the money and disappeared somewhere, we don’t quite know right? Is the same true for the other player? So Katie, you also put 1 in, is that your explanation as well?

Student: Same reasoning.

Professor Ben Polak: Same reasoning, okay. So everyone figured out what was going on, and once again, we think probably, let’s try, Steven if it had been $3 in the hat what would you have done?

Student: I would have taken it.

Professor Ben Polak: Taken it, all right. So we’re basically getting what’s predicted here out of the game. Let’s just see that more formally, that idea. So the idea here is that the players who move early on in the game should do something we’ve always talked about before. They should put themselves in the shoes of the other players, but here that takes the form of anticipation. They’re anticipating what players down the tree are going to do. So the key idea here is anticipation. What they’re going to do is they’re going to look forward down the tree; imagine themselves in the shoes of the later players; look at the incentives facing those later players; imagine that they do the best they can–those later players do the best they can –; and then walk backwards through the tree.

So the key idea here is to look forward to the end of the tree and work back: look forward and work back. That’s exactly the process that Katie and Justin described themselves as doing. Now in this tree, let’s have a go at this. So let’s imagine ourselves in Steven or Nate’s shoes, having found–as they did in fact–having found $1 in the hat. They then are making a choice between adding $1 which nets them a $1.50 or taking the $1 which nets them $1. 1.50 is bigger than 1, so we think they’re going to go this way. Everyone happy with that? Conversely, if they found themselves with $3 in the hat then, if they add $3 –they match–they’re going to get $2, they’re going to net $2; and if they take the money out they’re going to get away with $3. 3 seems bigger than 2 so we think they’re going to go this way.

Again, we’re assuming that these are their real payoffs. Okay, so from Player I’s point of view, since Player I knows that Player II will observe the choice before making her own choice, and since Player I can put him or herself in Player II’s shoes and anticipate this choice, Player I is essentially choosing between what? If she puts in 0 she gets 0, if she puts in 1 she knows II will match and she’ll double her money and get 1, and if she puts in 3, then she knows Player II will take the money and run, in which case our lender will have lost $3. So she’s choosing between 0, 1 and -3 and she’s going to choose $1, which is exactly what happened in the game. Everyone happy with how we did that?

So this idea of starting at the last player, the player who moves last, solving out what they’re going to and then working our way back through the tree has a name. The name for this is “backward induction.” I apologize for the business-school students in the room who have had this hammered into them too many times already so I apologize for them in advance. Backward induction. Now it turns out there’s precisely one controversial thing about backward induction, which is whether you should say backward induction or backwards induction. So I’m going to leave the “s” off and call it backward induction, but it’s up to you which you think the correct English is. Okay.

Chapter 2. Sequential Games: Moral Hazard [00:17:57]

So, since we’ve got this game here let’s talk about it a bit. This idea of the lender and the borrower–it’s not an unimportant situation. We’ve simplified it down to a very simple choice here, but it’s a very basic choice out there in the real world. It’s an important choice, after all, what keeps the economy running is the ability of lenders to lend money to businesses that then invest them profitability, and are able to make returns. So this underlies a whole bunch of stuff going on in the economy. So let’s talk about this and the first thing I want to talk about is, I want to point out that there’s a problem here. There’s a bad thing here, much like there was a bad thing in the very first class of the semester. In the very first class of the semester the bad thing was a Prisoner’s Dilemma, but here it’s a little different but let’s focus on it a while.

So the bad thing is we ended up here. Not a disaster: the lender doubled her money. The borrower was able to go ahead with the project on a small scale and make some money. The bad thing, however, is that we would have liked to end up here. Had we been able to have a large project funded with $3 and had the borrower repaid those dollars, so matched in effort or whatever that happens to be, then we’d have ended up at an outcome that is better not just for the lender–it’s obviously better for the lender–but it’s also better for the borrower. Does everyone see that? So the outcome we would like to have reached gets the lender 3 whereas she’s only getting 1 where we ended up, and it gets the borrow 2 where he’s only getting 1.50 where we actually ended up.

Why is it that we were unable through both of our pairs, both Justin and Nate, and Katie and Steven–why were we unable to get to here? Any takers? What got us, what prevents us from getting to this good outcome? Let’s go back to Katie a second and see if we can, go ahead. I’ll come it’s all right. So Katie was saying why she invested 1. I picked on Justin before, so here’s this good outcome, it’s better for you, it’s better for Steven who’s your borrower, why aren’t we getting to this 3, 2 outcome?

Student: Because if I play 3, him matching my money is dominated, strictly dominated.

Professor Ben Polak: All right, so the problem here is Katie would like to invest 3 if she knew that in fact Steven was going to match, but she knows Steven’s not going to match. Steven, in fact, is going to run off with the cash. It’s going to end up being spent at Starbuck’s or something. From Steven’s point of view he would actually like to be able to borrow the $3. He would like to be able to end up at this 3, 2 outcome as well. The problem is he knows that Katie knows, and Katie knows that he knows, that in fact he’s going to run off with the cash. Is that right? So we’re in a bit of a problem here. We’d like to get this good outcome but incentives are preventing us from getting there. What do we call this kind of situation? What do we call this kind of problem? Anybody? No takers?

This is called “moral hazard.” How many of you have heard the term moral hazard before? A number of people have heard of it. So moral hazard here is the problem. That the agent, in this case the borrower has incentives–will have incentives–to do things that are not in the interests of the principal. If we’re not careful, for example, by giving too big a loan, the incentives of the borrower will be such that they will actually not be aligned with the incentives of the lender and that will prove to be bad for the lender. But notice the existence of this moral hazard problem isn’t only bad for the lender. It ends up being bad for the borrower as well. Let me give you another example, in insurance there’s a classic, probably the classic moral-hazard problem.

If an insurance company insures me against having my car stolen, there’s a moral hazard problem in that I might, if the car is fully insured, I might now have the incentive not to bother to lock my car or to leave it anywhere on the street in New Haven, because I’m not bearing the cost. So we need to be able to–the insurance company needs to worry about that in writing an insurance contract for me. And in practice what the insurance company does is it forces me to take a deductible. It doesn’t allow me full insurance. It makes sure that some of the cost of having a car stolen is going to fall on me. That’s bad for me by the way because it means I can’t get full insurance. And notice that what happened here is rather similar.

In this case, both the lender and the borrower would rather have a big project, a big loan, and have the agent pay back that loan. They’d prefer this (3, 2) outcome, but they can’t get there. And therefore rather than having a full scale project, Katie or Justin only offered the borrowers a small scale project. So the idea here in this example was–what happened was we kept the size of the project or the size of the loan–you can think of this as the project if they’re borrowing something–small to reduce the temptation to cheat, to reduce the temptation not to repay the loan. So these problems are all over society but they’re particularly prevalent in situations involving lenders and borrowers. So a good question is how might we solve this problem?

A lot of you are about to go out and try and borrow money to set up your projects, run your new version of Facebook or whatever. And a lot more of you as we said before, are going to end up loaning money, perhaps someone else’s money probably, at least for a while. And we want to know how you might solve this moral hazard problem. We’ve seen one way to solve it. One way is for me to keep the size of the project small. What else could we do? Let me come down and try and have a bit of a discussion. What else could we do? Steven what else could we do? You were the borrower, what else could you do?

Student: Impose laws.

Professor Ben Polak: We could impose laws. Okay, so we could impose laws much like we tried in the Prisoner’s Dilemma, we could regulate this market. There’s going to be a little bit of a problem we’re going to run into in regulating the law, in using the law here. What’s the problem we’re going to run into here? We do have laws that regulate borrowers and lenders. What’s the law that regulates borrowers and lenders: bankruptcy law right? So we have law that regulates to some extent what lenders and borrowers can do. One of the things that we do in that law is we limit the degree to which we’re allowed to punish the borrowers. So bankruptcy law pretty much says we’re going to put some limit on how much Katie can inflict pain on Steven for running off with the lunch money. Steven can simply say I can’t repay, I’m bankrupt, and he’s more or less allowed to have a fresh start.

So the law would be good here but there’s limits to what we can do. We could go back to the Dickensian world of throwing Steven in jail for not repaying the loan, but it’s tough. There are down sides to that it turns out. So what else could we do? Yeah, I’m going to have to dive in there, so shout into the mike.

Student: Venture capitalist’s can impose restrictions on what the person can do with the money.

Professor Ben Polak: So you could try and say there’s only certain things you can do with the money. And, for example, to make that credible, what you might have to do is to say: show me what you’re going to do with the money, show me the receipts. Or another thing you could say is, I will only give you the money if you show me exactly how you’re going to spend this. That’s a good solution. Notice that you could regard that as changing the order of play. It’s a little bit like saying, let’s have the borrower have to move first, let’s have the borrower have to commit what their actions are going to be, how they’re going to spend the money before–you know–show the contracts before the lender lends them the money. That’s a good solution, but again there are limitations to this solution.

What are the limitations to that solution in the real world? Let’s go back to all of you who are thinking about making millions with your new Facebook, what are the limitations to having some contracts with the bank, with Chase, saying exactly how you’re going to–not only is it saying exactly how you’re going to spend the money but actually detailing it already with contracts ahead of time. What are the problems with that? Someone else who’s thinking of going into business here–let me pick on somebody a second. So what’s your name sir?

Student: Kelly

Professor Ben Polak: Kelly, so imagine you have some project, you’re going to borrow some money from Chase or Mr. Swenson to set up this fund, why might you worry about having everything written out contractually ahead of time?

Student: I mean then it restricts your freedom on what your ideas are, I guess.

Professor Ben Polak: Right, so one problem is just sheer lack of flexibility. If, in fact, the entrepreneur is not going to be flexible on how they actually run the project, there’s not much point of being an entrepreneur in the first place. There’s a limit to how much control you want to give to the lender. The lenders don’t really want to run the project for you. There’s also a severe problem of timing. It’s simply not going to be possible to specify all the expenditures of a project up front. You’re going to need some money to do some things ahead of time. So it’s a good idea–changing the order of play is a good idea–but there’s going to be limitations to this. What else could you do as the lender? What else could you do? Thinking of loaning some money in here, loaning some money to one of your classmates for some project. Let’s try here.

Student: Something like loan money over time in stages and observe progress.

Professor Ben Polak: Good, so one possibility is to break the loan up and give it–let it come in small installments. If you do well on the first installment I’ll give you a bigger installment next time. Notice that that’s a little bit like something we talked about in the very first week. It’s a little bit like taking this one shot game and turning it into a repeated game. It’s turning a one shot interaction into a repeated interaction. We’ll come back and talk about repeated interactions later on in the course. Again, notice that there might be some limitations to this. I mean I think it’s a good idea, and in general you should be trying to do this, but it could just be that the set up–for Nate to set up his new Facebook and for Steven to set up his new mousetrap factory–they just need a chunk of money at the beginning. There may be some limit to how little money you can give them at the beginning since they have to hire workers, and set up the factory, and make some big splash in advertising, and so on and so forth.

Chapter 3. Sequential Games: Incentive Design [00:29:50]

So again, it’s a good idea but there may be limitations here. What else could you do? Let’s have a look at the numbers on the board. Right now, the numbers on the board say that if Steven’s mousetrap project–were you the Facebook or the mousetrap?–if Steven’s mousetrap project goes well and it ends up being a big project with lots of money available. There are five units of money available in profit, 3 plus 2. The way we’ve designed things so far, our lender–I gave you these payoffs in the game–the lender is going to double her money. She’s going to get 6 back to net 3 and Steven’s going to get 2. So the division of that 5 is 3 and 2. But we don’t have to divide the money like that. If you’re a lender you may want to think about how you divide up the spoils of a successful project. So how else could you divide up those $5, that 3 plus 2, in a way that might be better for everybody? Nate?

Student: Give the investor $3.01 if they do that other choice and then take a little bit of a hit but that way you’ll still have something that’s pretty optimal to the other choices that we came to before.

Professor Ben Polak: I’m guessing you all didn’t hear that but that was right, so let me just say it. That’s exactly right. So what you could do here is you could redesign the payoffs of this project in the event that it’s successful. What you could do is take these 5 units of profit and instead of having (3,2)–let’s write that in a different color–we could give the borrower, the entrepreneur 3.1, leaving 1.9 to the lender. All right, so we could change the payoffs of this contract to be (1.9,3.1) in the event of a big project. Why does that help? Because now our borrower, Nate or Steven, were they to end up with a hat with $3 in it, they now will find it in their interest to match the $3 and get 3.1 rather than 3.

So what’s this an example of? This idea of changing the contract to give the borrower incentives to repay, incentives not to shirk, incentives not to disappear with the cash, is an example of what’s called “incentive design.” So the idea here is, in a lot of businesses, incentives are not given by God. They’re designed by the people who write the contracts. If you’re either the borrower or the lender, you should be thinking about writing a contract that’s going to achieve the end that you want. Now notice in this particular example, exactly as Nate said, the lender’s going to take a bit of a hit here, they’re not going to get a 100% return, but they’re ending up doing better than they would have done by giving a small loan. They’ve ended up getting 1.9 rather than 1, so in this particular example–it won’t always be the case but in this example–they’re taking a smaller share of a larger pie.

Sometimes a smaller share of a larger pie is bigger than a larger share of a smaller pie: not always but sometimes. The reason the pie is larger is we’re able now in an incentive compatible way–we’re able to go ahead and have a large project. So in this example, the smaller share of the larger pie actually ended up being bigger than the larger share of the smaller pie. So a smaller share of a larger pie can be bigger than a large share of a small pie, not always of course. This is just an example. Now let’s just be a little bit careful here. So in this particular example, even though this 1.9–this return that our lender is able to make by accepting a smaller share of a larger pie–even though it’s a bigger absolute return, it’s smaller in another sense. In what sense is it smaller? Let’s be careful here.

We’re tempted to say look it’s 1.9. 1.9 is bigger than 1. That’s got to be a good thing. But since some of you actually are going to go out into investment banks, and you might be posed this question, let me just make sure we don’t make a blunder that’s going to spoil your interview. So what might you be worried about as an investment banker about this deal, about this contract? Somebody else, anybody else? I’ll come down again, that’s all right. What might you be worried about? Who here, be honest, who here has been interviewing with either investment banks or venture capital firms, or private equity firms? Raise your hands. Oh come on there has got to be more than that. There was a hand. Where was the hand at the back? There was a hand way back here. No, yeah? What might be wrong with this project from your point of view? Can I get the mike on that side? This gentleman here, what might be wrong with the project from your point of view?

Student: If you could actually just find three small projects then you could still get a 100% returns just by dividing it up.

Professor Ben Polak: Good, you’re going to make lots of money for Morgan Stanley or whatever it is, good. Your name is? Brian, right so it’s a good ad to hire Brian at Morgan Stanley. So what Brian said was it’s true that you’re making a bigger absolute return here. This is a bigger absolute return. But this return up here is a 100% return on your capital. The rate of return on capital is higher making a 100% return on an investment of 1, than making a 1.9 return on an investment of 3. The rate of return is better in the smaller case.

So which is the right answer? Should we worry about the rate of return, the 100% rate of return or should we worry about the absolute return? If you’re in the business of investment banking, who thinks we should worry about the absolute return? Who thinks we should worry about the rate of return? Who thinks it depends? What does it depend on? (Keep me from falling off this thing.) Wwhat does it depend on? Somebody said it depends. What does it depend on? Depends is only a good compromise vote if you know what it depends on. Yeah, the gentleman here.

Student: Since an investment banking firm only has limited amount of money to invest, you want to see if you can get a higher rate of return investing elsewhere.

Professor Ben Polak: All right, so there’s two–I think you’re on the right line. So there’s two things to worry about, one is what is your supply of funds to invest and the other is–what’s the other supply you are worried about as an investment banker or venture capitalist firm, or a private equity investor? What’s the other supply coming your way? The number of projects coming across your door. The demand for your cash. The supply of projects. What we have to worry about here is what is the true opportunity cost of that money? If there were very few projects out there then you basically just want to go down the list of projects picking off absolute return, however, you go to the other extreme if there’s an infinite supply of projects, all coming in, all of which will offer a 100% return, then as our banker at the back told us, you should just go for three small projects rather than one large one.

But the question is, which is better? It isn’t as simple as saying, yes, we should always go for three small projects. There may not be three small projects out there that offer a 100% return. So what’s going to matter here is what’s the true opportunity cost of the capital, and that’s going to depend on what projects are available to you. This was just an example of incentive design, just one example of incentive design. Let’s talk about other examples of incentive design. Incentive design’s been a lot in the news lately. Who else tends to have incentive contracts other than borrowers? Who else out there tends to have incentive contracts? It’s a big topic in the news these days. So who has incentives written into their contracts? Way over here, let me run over at the risk of blurring the camera. So who else has incentives in their contracts?

Student: Managers

Professor Ben Polak: Managers all right. So CEO’s have huge incentive clauses in their contracts, sometimes in the form of options. Why do they have those incentive clauses in their contract? Well there’s two interpretations of this. One interpretation is, it’s just a bad thing and they’re just trying to screw the world. But there’s a more moderate view of this. The more moderate view of this is that those incentive clauses in their contracts are attempts to align the interest of the managers with the interest of the shareholders, just as this contract is attempting to align the interest of the lender and the interest of the borrower. So by managers I took him to mean CEOs, but what other kind of managers recently in the press, very recently, this weekend, have been talking about incentive contracts? Way over there, I’m caught on the wrong side as usual. Can I squeeze through? I’ll try and squeeze through. Can I squeeze through? Sorry. Where was it, where was the answer to my question? Here we go.

Student: Managers of baseball teams.

Professor Ben Polak: All right, so managers of sports teams also have incentive contracts these days: incentives to achieve certain levels. Here it’s not quite clear to me exactly what’s going on. We call those incentive contracts but you might want to worry a little bit. I mean do we really think that the manager has an incentive to lose otherwise? So part of what’s going on with incentive contracts isn’t actually incentive contracts, part of it is about sharing the risk. Particularly in the kind of contracts that have been written about over the weekend with Joe Torres’ contract for the Yankees, which he turned down, is partly perhaps to give him incentives to try and win, but perhaps it’s more to do with sharing the risk among the general manager and the manager of the team if they don’t make it to the playoffs.

But all right, they’re still incentive contracts and we see incentive contracts elsewhere, and we’ve seen them since the Middle Ages. So in the Middle Ages, incentive contracts took a particular form, and the form was called two things, piece rates and in agriculture it took the form of share cropping. What are piece rates? Somebody, what are piece rates? Anybody who knows what a piece rate is? What is share cropping? Somebody must be from a farming state. Somebody out here, the guy in orange.

Student: Share cropping is where the landowner takes a portion of the farmer’s crop at the end of the farming season.

Professor Ben Polak: All right, so share cropping is a contract whereby part of the output of the farm goes to the landowner and part is kept by the farmer. Going back to the Middle Ages, part of the output would stay with the peasant and part of it would go to the lord of the manor. Piece rates are a similar idea. Rather than paying wages to workers, just bundles of cash, the amount which they’re paid depends on output. So piece rates would say if you produce seventeen yards of cloth then you’ll get paid the revenue that’s derived from three of those. Both of these forms of contracting are there partly to create incentives. They create incentives because they create the incentive for the farmer to increase yields and they create incentives on the part of the worker to increase output. So piece rates are a big part of the story.

Incentive designs are a pretty big topic and we don’t really have time to go into it much in this course. But it’s worth mentioning that, in fact, the Nobel Prize that was won last week by those three economists we talked about was partly about incentive design. So it’s regarded as an important enough topic, the design of incentives across society as a whole to have been worthy of winning the Nobel Prize, and my guess is that there’s another Nobel Prize that’s going to come out in this same area in the next five or ten years.

Just to make it clear, you can have problems: you can write incentives badly. It’s a classic story that you’ve all heard of I’m sure, that often in the provision of public goods, let’s say bridges or highways, or tunnels, the incentives for the construction companies tend to be rather badly written. They’re meant to have incentive clauses but they never seem to quite get them right. It’s easy to get these incentives wrong. Just to prove this just doesn’t happen to management, let me just tell a story against myself. So a year ago, a year and a bit ago, I was picking fruit with my then three year old daughter and I wanted to make sure that we got some fruit home to eat for dinner, and she didn’t just eat all the fruit as we picked it. These were raspberries. So I decided to have a rule that sounds efficient. I said any item of fruit that gets picked that’s in good condition we’ll take it home for dinner, but if it’s squashed, then we’ll eat it then and there while we’re picking the fruit. So my three year old proceeded to go along the bushes of raspberries, picking the raspberry, turning around to me and saying “squashed” and putting it in her mouth: “squashed,” “squashed,” which shows that a three year old knows more than a professor about moral hazard at some level.

Chapter 4. Sequential Games: Commitment Strategies [00:44:29]

So one thing we can do is worry about incentive design in the design of the contract. There’s another thing we can do here. What’s the other thing we can do here? We’re missing out on something very basic. What is it that our borrowers Steven and Nate can do? To put it more generally, for all of you, if you go out and start a company, and you borrow some money to start your company when you leave Yale, what are you going to have to do? What’s the borrower–what’s the lender, what’s the bank going to insist–what is it going to insist on? Anybody? They’re going to insist on collateral. What is collateral? I’m somewhat charmed to see how few of you are going to end up borrowing money, but for those of you who haven’t got trust funds, you’re going to end up borrowing money. So what is collateral? Yes, what’s collateral?

Student: You give someone the whole–You let the bank hold onto something.

Professor Ben Polak: Right, you let the bank hold onto something in the event that you default on the loan. You’re going to let the bank hold onto something in the event that you default on the loan. Now for most people here, what are they going to post as collateral? Let’s assume that we’re back to our original contract here–we can’t do this–we’re back to our original case. Wwhat are most of you going to end up using as collateral if you borrow money? Everyone’s saying a house, I’m guessing that for most of you –I’m not going to look around the room because I don’t know how many trust fund babies I’ve got in the room–but for most of you, it’s going to be what? It’s going to be your parent’s house. So why is it that your parent’s house being posted as collateral is going to prevent you from disappearing to Mexico with a loan?

Well let’s have a look at how that changes the payoffs. Now if you take the money and run, you disappear to Mexico, but you end up with $1 minus a house here, and $3 minus a house there. Of course, we’re relying here on you liking your parents enough not to mind about the fact that they’ll be sleeping in the rain, but as long as that’s the case, I think with high probability you’re now going to decide to return the loan, at least if you can. Is that right? Now notice that the way in which this collateral works. There’s a subtlety to how this collateral works. You might think that the way in which collateral works is that now the lender feels safer, because even if you end up defaulting on the loan, at least they get this house. That’s true. There’s some truth in that, but that’s really not the key here. Frankly, most bank lenders don’t want your parent’s house. Your parents might have very nice houses but this isn’t a good housing market right now and they don’t particularly want to see your parents on the streets.

The way in which this works is not so much that it gives an extra positive return to the lender, the way in which it works is it gives an extra negative return to the borrower. So let’s just do this backward induction again with the house in place. So with this house in place, as before, you’ll return the loan up here, but now provided you like your parents enough, you would also return the loan here because you prefer $2 to $3 minus your parent’s house, in which case you’ll get a loan of 3. So the way in which the collateral worked was by hurting you enough (or in this case hurting your parents enough) in the event of default, in such a way as to change your behavior. But notice it helped you. So here’s a subtlety coming in here. By issuing collateral, we actually lowered the payoff of the borrower at certain end points of the tree. We lowered the payoff of the borrower at certain end points of the tree. But as a consequence we were able to end up at an outcome that was better for the borrower.

Sometimes having lower payoffs can actually make you better off, and the reason it makes you better off, is it changes the behavior of other players in the game in such a way as may benefit you; in this example, by inducing the lender to give you a bigger loan. So the way the collateral works is it lowers your payoffs if you do not repay, but it leads to you being better off. The reason it makes you better off is it changes the choices of others in a way that helps you. We’re going to see lots of examples of that in the next couple of days. Now collateral is an example, we talked about incentive design, collateral is also an example of something larger. The larger thing is the idea of commitment. Collateral is a commitment strategy. It commits you to repaying the loan.

I want to give another example of a commitment strategy. I think this idea of commitments is an important one, and we have just about time to do it. So I wanted to go completely away from our context of our lender borrower game a second and look somewhere else entirely, so we get rid of this for now. I want to take us back to 1066, what happened in 1066? What happened in 1066, I know this is America and no one gets an education anymore, but still, what happened in 1066? The Norman Conquest took place in 1066. So the Norman Army invades from Normandy led by William the Conqueror and lands on the Sussex beaches at Hastings. What I want to imagine is a game involving the Norman Army, or William the Conqueror in particular, and the Saxon Army.

So here’s the little game. So we’re in the area of medieval military strategy, and here is the game. We’ll start with the Saxon’s here, the Norman’s have already invaded, so I guess the Norman’s have already moved but here’s the Saxon Army making a decision. They’re deciding between fighting and the important medieval military tactic known as running away. Of course the Normans are going to know whether the Saxons are actually going to charge down the beach and attack them, and they are going to have to decide whether to fight or run away. Let’s put the payoffs–now leave some space below here, leave some space down here on our notes–let’s put the payoffs in. So I’m going to put the Norman payoffs first and the Saxon payoffs second. Norman payoffs first, Saxon payoffs second.

So the payoffs are going to be like this. If both sides fight, a lot of people end up dead and we’ll call that (0,0). If the Saxons fight and the Normans run away, the Saxons get 2 and the Normans get 1 because at least they’re alive. If the Saxons run away and the Normans fight, then the Normans get 2 and the Saxons get 1. I just flip things around. And if both lots run away then ultimately it’s good for the Saxons. So this is the game. Let’s be careful, I’ve put the Saxon payoff second. If we analyze this game by backward induction we can see that the Normans are in trouble. Why are the Normans in trouble? Because if they are attacked by the Saxons then fighting yields 0 and running away yields 1, so they’re going to run away. Conversely, if the Saxons run away then in fact, the Normans will stay and fight because it’s easy to stay and fight against people who are running away. Even I can do that. So they’re going to go for 2 rather than 1.

But now the Saxons face this choice, they know that if they attack the Normans the Norman’s will run away and the Saxons will get 2. But if the Saxons run away then the Norman’s will stay and fight, and the Saxons will only get 1. So the Saxons are going to end up fighting. Everyone see how we did the backward induction? So this is not a good situation if you happen to be the Norman commander, if you’re the guy who led these troops across the channel. So what did William the Conqueror do? He burned his ships. So let’s now make this game a little bit more like the real historical situation, here’s a move that William the Conqueror, the King of the Normans can do. He can either not burn his ships or he can burn them. If he burns them then the game is rather different.

Once again, the Saxons can decide whether to fight or run away, but now the Normans really don’t have any choice. They have to stay and fight because swimming back to France wearing armor is difficult. So the payoffs become (0,0) and (2,1) as before. So we know now that the Normans are going to stay and fight. That’s all that’s available to them. I’ve put it that they don’t even have the option of running away. If you want, you could put running yields them minus infinity or whatever it is to get drowned. So now if the Saxons, if the Norman’s have burnt the boats, the Saxons know that if they fight the Saxons will get 0, if they run away they’ll get -1, so now they’ll run away. Then from William the Conqueror’s point of view, what’s he choosing between? He knows that if he doesn’t burn his boats the Saxons will fight, and his troops will run away. But if he does burn his boats, the Saxons will run away and his troops will fight so he wants to burn his boats.

Everyone okay with the example? Now this is probably the classic example of a commitment strategy. William the Conqueror, in this example, or Cortez in perhaps the more famous example, burned the boats to get rid of some strategies, to get rid of some choices altogether. And you might think how can getting rid of choices make me better off? The reason, in a game, particularly in a sequential game, in a game generally, the reason getting rid of choices can make me better off is it changes the behavior of people on the other side of the game in such a way as benefits me. In this particular example, it makes the other side less likely to fight. So once again this is a commitment and the way it works is–this example of a commitment is actually to have fewer options and it changes the behavior of others. If it didn’t change the behavior of others it wouldn’t be worth doing.

So let’s just talk about this a little bit more. There’s another famous example of a military commitment strategy. It occurs in the movie Dr. Strangelove. How many of you have seen the movie Dr. Strangelove? How many of you have not seen the movie Dr. Strangelove? Good homework exercise for the weekend: rent the movie Dr. Strangelove. It’s a very good movie. Stanley Kubrick I think. So in Dr. Strangelove, there’s a famous commitment strategy. What is the famous commitment strategy in Dr. Strangelove? Somebody? Let’s get a mike. Ale can we get a mike down here? There’s one on the way. The guy in white, there you go.

Student: They lose radio contact with the base, which means that they’re going to end up dropping the bomb.

Professor Ben Polak: All right, that’s true. I was thinking of something more deliberate. What’s the more deliberate–the guy in red–while you’re there, somebody in red here. Here we go.

Student: The Soviets construct a doomsday device so if they’re attacked they automatically launch all of their nuclear weapons.

Professor Ben Polak: Good, so the worry that the Soviets have is they think the Americans might attack them, and they worry that the Americans might attack them thinking that they won’t retaliate, because they won’t really have any incentive to retaliate once their cities are destroyed. So they construct a device they called the doomsday device that will automatically launch nuclear missiles back on the Americans if the Russians are attacked. It’s a commitment device to make it credible that they will actually respond were they to be attacked by the crazy Americans. So there’s a mistake, however, that the Russians make. What is the mistake that the Russians make? The guy in the Yale sweatshirt here.

Student: They don’t tell anyone about the machine.

Professor Ben Polak: Right, they don’t tell the Americans that they’ve built the doomsday device. So this is really an important idea hidden in what’s really a great movie. Commitment strategies are a good idea. They can make things credible. But the only reason that having fewer strategies is good for you is because it changes the behavior on the other side. In this case, it changes the behavior of the Saxons. It doesn’t work if the other side doesn’t know about it. It’s crucial that the other side knows. So the other players must know. Just to hammer this home, notice that the expression in the English language is “burning your boats.” It’s not “going over to your boats and quietly drilling a little hole in them,” it’s burning your boats. You want the other side to see this big bonfire on the beach and to know that the boats are no longer available. Just as in Dr. Strangelove, it’s crucial, and it’s a terrible mistake for the Russians not to tell the Americans that they’ve built this machine.

So knowledge here is crucial, knowledge of the other side. Now I should just complete the history of this. This didn’t work in the case of the Battle of Hastings. The Saxons attacked anyway and it just turned, it was a big blood bath, and the Normans really only won this battle because somebody got a lucky shot and hit King Harold in the eye. So this is a good story but as a military tactic not actually a true story. I’m adding that in, because otherwise I’m going to get hundreds of letters from people who watched the video. Now we’ve had a bunch of lessons today, but I’m not done, I want to play one more game. Before I do though I want to make sure we understand–we’re going to play another game so everybody stay put. I want to make sure. There’s a bunch of lessons on the board, and I want you to understand what’s important and what’s more important. Everything is important, but some things are more important.

Chapter 5. Sequential Games: Backward Induction Is Really Important [01:01:06]

So the idea of commitment is an important idea, and the idea of incentive design is an important idea, but there’s one idea here that is more important than anything else, and that idea is backward induction. I’ll put a little sun around it okay. How important is backward induction? Backward induction is the most important thing I’m going to teach you in the second half of the semester. It’s probably the most important thing I’m going to teach you in the whole semester, and it may be the most important thing you’re going to learn at Yale. It’s the answer to all questions. If I ask you a question, and, as happens from time to time, you’ve been sleeping in class and you wake up startled because I’ve now asked you a question, and that camera is on you, and you don’t know what the question is, then the answer is backward induction.

I’m stressing this because six weeks from now, whatever it is, in December, there’s going to be a final exam and I guarantee now that there will be a question on the final exam that involves backward induction, and I guarantee that at least one person in the room will forget to solve it by backward induction. So are you going to use backward induction on the final exam? Yes, you’re all going to remember it. Okay good, now we’ll play one more game just for fun. But before we play one more game, I’ll do one other thing, sorry one other thing before we play one more game, I apologize. The one other thing I want to do here is I put up a tree here but I forgot to tell you anything about the tree. This was a tree, and I would be remiss if I didn’t tell you a few things about trees.

So there’s a certain amount of jargon that goes along with trees, and I want you to go home and read the textbook and know what that jargon is. So these spots, these dots in the tree, these things are called nodes. And in fact the end points of the tree where the payoffs are, these are also called nodes but they’re called end nodes. These branches in the tree, the branches, here they are, the branches are called edges. So a tree consists of nodes, edges, and these missing dots are the end nodes. The end nodes are associated with payoffs. Do we need to write that? That’s obvious right: they’re associated with payoffs. And every other node belongs to somebody whose turn it is to make a decision at that node. These other nodes, the ones that aren’t end nodes are called decision nodes.

It’s perfectly possible, for example, in our Norman Conquest game, for somebody to have several nodes. Somebody could move once and then later on again. There may be also nodes that are never reached in a game. One last piece of jargon, a route through a tree, from the beginning to an end is called a path, a way of getting through the tree from beginning to end. So we’ll have a pop quiz on the jargon later and now let’s play the last game I want to play. We’ve got ten minutes right? Everyone understand the jargon? I want to play a game now.

So the game we’re going to play is a game we’re going to call the “Hungry Lion Game.” And what I’m going to do is I’m going to pick on a row of the class, actually I’m going to pick on this row of the class, and this row of the class, everyone can see this row? From up there, lean over the balcony. This row of the class, we’re going to imagine that everybody in this row is a lion. Do they all look like lions? Not much, but never mind. Pretend their lions: suspension of disbelief. They’re all lions and they’re all hungry except for one person, who’s this guy at the end whose name is? So Alex is a sheep. Alex had the misfortune of wandering into this pack of–pride of lions and is liable to get eaten because they’re hungry. However, one thing stands between Alex the sheep and being eaten. The one thing that stands between him and being eaten is that in lion society, the only lion who is allowed to eat the sheep is the head lion, the big lion whose name is Ryan. Ryan is the head lion here.

Only Ryan can eat Alex the sheep. No other lion can eat Alex the sheep. Now there’s a catch if Ryan eats Alex–if Ryan the lion eats Alex the sheep. The catch is, that if Ryan the lion eats Alex the sheep, then Ryan the lion will fall into a postprandial stupor, fall asleep, at which point the second largest lion whose name is Chris can eat Ryan. Now notice Chris can never eat the sheep. He can only ever eat the lion, the big lion, and only then if the lion has eaten the sheep and falls asleep. This is very strict. Lion society is very hierarchical like England. If, however, Ryan the lion eats Alex the sheep, and is then eaten, falls asleep and is then eaten by Chris the lion, then and only then he will fall into a stupor at which point he can be eaten by Isabella, who’s the third largest lion and so on.

Everyone understand the game? No questions, no talking among yourselves. Write down what you would do if you were our key decision maker, who is Ryan the lion. But before we write it down can I get Alex to stand up a second, stand up, camera on him, this is the menu. Thank you. So you have to write down in your notepad whether, you, Ryan the lion, in Ryan’s shoes or paws or whatever, whether you would eat Alex the sheep. Everybody written something down? Let’s ask Ryan the lion what he’s going to do, so Ryan are you going to eat Alex the sheep?

Student: No.

Professor Ben Polak: He says no, he’s not going to eat it, why not?

Student: Well then because I would be eaten as well.

Professor Ben Polak: Then he’d be eaten. Well let’s see if he would have been eaten. So had he eaten Alex the sheep, then he would have fallen asleep, and then he’d have been in danger of being eaten by Chris. Chris would you have eaten Ryan?

Student: No.

Professor Ben Polak: So he wouldn’t have eaten him in fact. In fact, it looks like Ryan would have got a free lunch. There are no free lunches in Economics, so something’s wrong. Why is Chris worried about it? Chris is worried about it because–Chris is worried about Isabella eating him, and what am I doing wrong? What’s wrong with this analysis? By the way, before we say what’s wrong, how many of you in Ryan’s shoes would have eaten the sheep? How many of you would not have eaten the sheep? Okay, what am I doing wrong? I’m in totally the wrong part of the room. I shouldn’t be here at all. I shouldn’t be at all in this part of the classroom. I can’t get around. Wait a second. Excuse me. I should be,– which row were my lions? Where’s my lion row?

I should be way over here with the baby lion, whose name is Agay. So Agay is a little baby lion–very cute–but Agay, if he gets a chance to eat is going to eat. Why is he going to eat? Because there’s no one to eat him. So if Agay gets the chance to eat the second babiest lion whose name is John, then Agay will eat. So if John gets a chance to eat Ben, is John going to eat Ben? John are you going to eat Ben? No, John’s not going to eat Ben because John knows he’ll get eaten by Agay. So Ben, if you get a chance to eat Vidur, are you going to eat Vidur? Wait a second. We just argued that if you eat then the second largest lion whose name was John is not going to eat you. Yes. So we know the littlest lion is going to eat. So the second littlest is not going to eat. So the third littlest lion can eat safely. So the fourth littlest lion should not eat. So the fifth littlest lion should eat. We’ve got eat, not eat, eat, not eat, eat, not eat, eat, not eat, eat, not eat, eat, not eat, eat, not eat, eat, not eat, eat, not eat, eat, not eat, eat, is that right?

So in fact he should have eaten him, but the more important point is how should we have analyzed this game? What should we have used? Backward induction. How many of you used backward induction? Be honest, so what’s the point here? The point is this: five minutes ago I said the most important thing you’ll ever learn at Yale is backward induction, then I distracted you with some fairly irrelevant nonsense, and five minutes later nobody used backward induction. So the main lesson from today, this is where we’re going to pick up on Wednesday is to use something, what are we going to use? Backward induction. All right we’ll see you on Wednesday.

[end of transcript]

Back to Top