Financial Markets (2011)

ECON 252 (2011) - Lecture 7 - Efficient Markets

Chapter 1. Swensen’s Lecture in Retrospect and Manipulations of the Sharpe Ratio [00:00:00]

Professor Robert Shiller: And I want to talk today about efficient markets, which is a theory that is a half-truth, I will say. Before I start, I wanted to just give a few thoughts about David Swensen’s lecture last period. Let me say, first of all, the Efficient Markets Hypothesis or the Efficient Markets Theory is a theory that markets efficiently incorporate all public information. And that, therefore, you cannot beat the market, because the market has all the information in it. You think you’re smarter than the market, that you know something? No, the market knows more than you do. And you’ll find out that the market wins every time. That’s the Efficient Markets Hypothesis. So, it’s a very far-reaching hypothesis. It means that, don’t even try to beat the market. That was a very rudimentary introduction to today’s lecture.

But here I brought in David Swensen, who is claimed to have beaten the market consistently since 1985, and dramatically. And so, what do we make of that? That’s the subject of today’s lecture. By the way, after class one of you came up–thank you, it was nice, I don’t know where you are–one of you came up and thanked Swensen for his scholarship at Yale. Yale now has need-blind admissions for the world. And so people, not just from the United States, people are helped out, so that people who have managed to meet the high admission standards here, makes it possible for them to actually come here. And that’s substantially David Swensen who did that, because it’s not just the generosity of the university. They have to have the money to do it. And so, somehow he seems to have made it.

Now, I know that there are still many cynics–the Efficient Markets Hypothesis has a lot of adherents still. And as I say, it’s a half-truth. So, some people will say, well, Swensen was just lucky. And I say, how could he have been lucky for 25 years in a row? Well, not every single year, but pretty much. And they say, well, you’re picking the one guy out of millions who is just the luckiest. So, those arguments are made. Anyway, one of you asked a question, which I thought it was very good, at the end. And that is why, in all of my discussion about Swensen and all of his talk he never mentioned the Sharpe ratio? Because as we said, the Sharpe ratio corrects for risk taking. That was one of our fundamental lessons. When we showed you the Efficient Portfolio Frontier, and the tangency line. You can get any expected return you want at the expense of higher uncertainty. You do a very risky portfolio, and you have high-expected return, because of the risk. If the risk is measured right.

But I think that’s a good very good question. I caught myself not correcting for it, I just said Yale’s portfolio had a high return. I didn’t correct for standard deviation of return. So, David Swensen, in his answer, as you recall, essentially said he doesn’t believe in Sharpe ratios, because we can’t measure the standard deviation. The Sharpe ratio is the excess return of a portfolio over the market, divided by the standard deviation of the return. And that scales it down, so if the excess return is very high but also has a very high standard deviation, that shows they were just taking risks. And so the Sharpe ratio would reveal that. But Swensen said, I don’t think that you can measure the standard deviation of return. Isn’t that what is answer was? You were here. I may not be quoting him exactly right.

So, why wouldn’t you be able to measure the standard deviation of returns? He gave a reason, which was that, well, when you’re looking at a broad portfolio like Yale’s, a lot of the things in there are private equity–that means privately held, so it’s not traded on stock exchanges. Or it’s real estate. Real estate is only traded every 10 years or 20 years, and so who knows what it’s worth? All you have is an appraisal, but that’s just some appraiser’s estimate, so the standard deviation would be artificially low. He’s right about that, but I think there’s even more to that. I know there’s more to this. There’s a literature on this.

The point that I wanted to make is that you can do a–suppose you’re managing money. And suppose the world out there is evaluating you by your Sharpe ratio. And suppose you have no ethics. You just want money. This isn’t so obviously criminal, this is not criminal I suppose. So, you say, I just want to have the best Sharpe ratio for a number of years running. I’ll get more and more people that will put money in my investment fund, and eventually I don’t care what happens. I’ll move to Brazil or something. Some foreign–pick out one. I want to get out of here with the money. So, all I have to do is fool people into thinking I have a high Sharpe ratio for a while. So, what do I do? Well, there’s an interesting paper on this by–there’s a lot of papers on this, but I’m going to cite one–by Professor Goetzmann and co-authors, here at Yale. It’s actually Goetzmann, Ibbotson, Spiegel, and Welch. Maybe I’ll put all their names on. Roger Ibbotson is a professor here, Matt Spiegel, and Ivo Welch.

What they did is they calculated the optimal strategy for someone who wants to play games with a Sharpe ratio. So, you want to fool investors and get a spuriously high Sharpe ratio. And they found out what the optimal strategy is. And that is to sell off the tails of your distribution of returns. So, if your return distribution looks like this–this is return, OK? And you have a probability distribution; say a bell-shaped curve, OK? And so the mean and standard deviation of this would be the inputs to the Sharpe ratio. But if you’re cynical and you want to play tricks, what you can do is sell the upper tail. These are very unlikely good events. Sell them and get money now, and then double up on the lower tail. So, you push the lower tail to something like that, and you wipe out the upper tail, so it goes like that, all right? So, that means you’ll get money, because you sold the upper tail. You would do that by selling calls–we haven’t talked about options yet–but you can do it by selling out-of-the-money calls. And you would do this by writing out-of-the-money puts.

OK, but what you do it you make it, so that if there’s really a bad year, it’s going to be a doozer bad year for your investors. And if there’s ever a good year, then, hey, you won’t get it. But these good or bad years occur only infrequently. So, in the meantime, you’re making profits from these sales and you have a high Sharpe ratio. But little do they know that you sold off the tails. And so, nothing happens for many years and you just look like the best guy there. So, it turns out that this is not just academic. There was a company called Integral Investment Management that did something like this strategy. It was a hedge fund. So, it was Integral Investment Management. It did something like this, by trading in options. And, it got lots of investors to put millions in them.

Notably, the Art Institute of Chicago put $43 million into this fund and its associated funds. And then, in 2001, when the market dropped a lot, Art Institute of Chicago was wiped out. They lost almost all of their $43 million. And so, they got really angry, and they sued this company. Because they said, you didn’t tell us. What have you been doing? And then, the company pointed out, in its defense, that it actually said somewhere in the fine print, that if markets go down more than 30%, there would be a problem. And somehow, nobody at the Art Institute read that or figured it out. They thought the guy was a genius, because this company had the highest Sharpe ratio in the industry, all right? You see what they’re doing? They’re playing tricks. They’re making it look like there’s less risk than there really is. And there’s a strategy to do that.

It didn’t end well for Integral Investment Management, because the Art Institute of Chicago managed to stick them on other things–they disclosed it. They told people that they were doing this strategy. The artists didn’t figure it out. But there were other dishonesties that they nailed these guys on.

What Goetzmann and his co-authors did is, they showed that you can play tricks with the–you can play a lot of tricks in finance. But one of them is to play a trick with the Sharpe ratio. But their trick was very explicit. It involved particular portfolio composition involving options, and any professional would immediately know that that’s a trick. And it didn’t work for these guys; they were too aggressive in their manipulation. But you can do subtler things as a portfolio manager to get your Sharpe ratio up. Instead of manipulating with special derivatives positions, you could just buy companies that have large left tails. They have a small probability of massive losses, right? And you haven’t done anything but pick a stock. And nobody knows whether it really has a small probability of massive losses, and you could systematically invest in that. And then you’d have a high Sharpe ratio for a while, and then you’d just blow up and lose everything eventually.

I was thinking of an example from recent news. What about the strategy of investing in Egyptian companies that are tied to Mubarak? It might have looked very good for a long time, right? But the companies might have been underpriced, because people sensed there’s some instability in Egypt. And look how fast it came on, right? I don’t know what the outcome will be at this point, but it just happened–bang. That’s a tail event, right? You might look at Egyptian securities and think everything is stable and fine, it’s been 30 years, nothing has happened. But someone knows or suspects that there’s something maybe unstable. And you, as an investor, wouldn’t know that by looking at the numbers.

What I’m getting at is really what is the essence of Swensen’s skill or contribution? It’s not his ability to manipulate Sharpe ratios or numbers like that. I think what it has, to me, in my mind, is something to do with character and his real self and his real objectives. And this gets at what people in finance are really doing. I think that when you take a finance course in a university, you may not get a proper appreciation of how one, in a career in finance, develops a reputation for integrity. Nobody can really judge what you’re doing as an investor, because they can’t judge it from the statistics. Even though we’ve developed this nice theory about Sharpe ratios and the like, you end up judging the person and what the person’s real objectives are. I’ll probably come back to that theme again.

Let me also say that’s about doing the Goetzmann–you understand the strategy? It’s like investing in securities with time bombs in them. They’re going to go off eventually, you don’t know exactly when. And they look good for a while, but they’ll blow up. What does the law say about this? Well, the law in the United States and other countries emphasizes that an investment manager must not fail to disclose relevant information about a security. And it has to be more than boilerplate disclosure. For, say Integral Investment Management, you could write up a prospectus and then say, but of course past returns are not a guide to the future and something could go wrong. That’s a boilerplate disclosure, because people think, well, that’s what everybody says. The law says that you have to actually actively disclose. If there’s something that’s relevant that would make your statistics misleading, you have to get their attention and explain it to them. That’s the law. So, I think it’s laws like that, and it’s the development of–it’s not something that we specialize in academia. Well, we are, we’re trying to develop character I suppose. But it’s not just Sharpe ratios. And, I think that tendencies to rely on numbers like this has led to errors in the past.

Chapter 2. History of the Efficient Markets Hypothesis [00:16:06]

So, let me go more directly into today’s lecture. It’s about the Efficient Markets Hypothesis, which is, to me, a fascinating theory, which is not completely true, which makes it all the more interesting. The first statement that I could find–I’m interested in the history of thought–so, the first statement of the Efficient Markets Hypothesis that I could find was in a book by George –

[SIDE CONVERSATION]

Professor Robert Shiller: George Gibson, who wrote a book in 1889 called The Stock Exchanges of London, Paris, and New York. And, he wrote, I’m quoting George Gibson, 1889: “When shares become publicly known in an open market, the value, which they acquire there, may be regarded as the judgment of the best intelligence concerning them.”

He described the stock market as a kind of voting machine where people vote. If you think a share is worth more, you vote by buying it. If you think it’s worth less in the market, you sell it. And everybody in the world can do that. It’s open to the public. So, the smartest people get into it, and then they soon make a lot of money doing it, so they have a lot of votes. So, the smarter people have more votes. And you’ve got everyone there. If anyone has a special clue, they go right in and they buy if it’s positive, or they sell if it’s negative. So, the smartest people go right in there and aggressively affect the value. Until it’s right, and then there’s no there’s no incentive to buy or sell.

The other thing about Gibson–I should have copied the quote, but as I remember from the book, he says something ”in our modern electric age, information flows with the speed of light.” I say, what is he talking about in 1889? Well, you know what he’s talking about. The telegraph. And, in fact, they had ticker machines. They were electronic printers that would print out stock quotes. So, they were really in the information age by 1889. So, this is what happened, Gibson said, you can’t beat the market. It’s just smarter because–it’s like Wikipedia is smarter than any one of us, right? Because it puts together all the thinking of all the people. Well, they had Wikipedia of a sort, because they had the stock market. They had the price. So, that is the statement that–he didn’t use the word ”Efficient Markets.” Actually, tried to find the origin of “Efficient Markets.” Sometimes in the 19th century, people would say ”Efficient Markets.” But it wasn’t a cliché yet; it wasn’t a phrase that would be recognizable. Even in this context, they would use it. But it was not a name for a theory yet.

The next Efficient Markets theorist–and I put this on your reading list–Charles Conant, who wrote a 1904 book called Wall Street and the Country. I put one chapter on that on the reading list, because it was a statement of the Efficient Markets Hypothesis, which was remarkably well written, I thought. He starts out the chapter by pointing out that a lot of people think speculation is a kind of gambling, or a kind of evil. Speculating on the stock market, that sounds like some wild activity that ought to be ruled out. Then he said, how can that possibly be true? The stock market is a central institution of all modern economies. To think that it’s just gambling just defies common sense. But then, he goes on and describes what it is that it does. And there’s some really nice passages in Conant’s book. In one passage he says, suppose for a moment that stock markets of the world were closed, what would happen? And he said, no one would know what anything is worth; no one could make any calculated decisions. He said that, in fact, capital moves around from one industry to another in respect of the prices that are quoted in these markets. And if you didn’t see the prices, you would be blind.

It’s often said about the Soviet economy–which did not have stock markets or financial markets–that they relied on prices in the rest of the world. Nobody in the Soviet Union could plan very well, because they didn’t know what anything was worth. But at least they had the rest of the world, and they thought, well that’s an approximation to what prices ought to be in the Soviet Union. So they were really relying on these.

I just wanted to reiterate a little bit about the intuition of Efficient Markets. The idea is that if you trade securities, the advantage to being there a little bit ahead of anyone else is enormous. If you know five minutes before the other investors about some good news or bad news, either way it doesn’t matter–you know it five minutes earlier, you jump right in and trade. You trade ahead of them and prices haven’t changed yet, you make money. So, that has created an industry that speeds information.

The first such industry that I would tell you about is Reuters. Mr. Reuters, I think in the 1840s before the telegraph, created a financial information service using carrier pigeons. You know, these are birds. And so, when he was in London with some information, they had carrier pigeons that were brought from Paris, and as soon as new information came out, they would tie it to the foot of the little bird and they’d let it go, and it would fly to Paris. It would go to its roosting place, the message would be read, and subscribers would be notified. And that was no joke, that really worked, because you would have information hours or even days before everyone else in Paris. So, you could make a killing. So, Reuters today, it’s now called Thomson Reuters, is still in that business. The carrier pigeons were a brilliant idea. But they had to keep up with the times, because, shortly thereafter, telegraph was invented and pigeons no longer were the leading technology. But maybe that was the beginning of the information age, with pigeons.

Now we have beepers, and we have the Internet. A beeper is something you can carry in your pocket that beeps when there’s financial news. So, suppose a company makes an announcement that that it has, say, a new drug that’s successful in trials. As soon as they make the announcement, it then goes out electronically everywhere, and the beepers start beeping. And all the investment analysts, they drop their morning coffee, because they know they got to act fast. So, you get this new announcement. Now it’s t plus 20 seconds. He’s got his drug specialist on the phone, OK? What does this mean? Quick, how much is it going to go up? And so the guy says, I don’t know, first thought, maybe it’s going to go up $2 a share. OK, it’s only gone up $1 a share; I’ll buy right now. And now it’s two minutes after the announcement, and then the analyst says, I’ve thought about it a little bit more. No, I only think it’s $1.50 a share. This is homing in, and so the price is jiggling around rapidly as all this is happening for a few minutes, and then it settles down. Because after–I may be exaggerating–after 10 minutes, they’ve kind of gotten it, they’ve kind of figured it out and it’s reached its new level. They’ll be thinking about it the next morning when they’re taking their shower, and they’ll get a better and better, more refined idea of what the price is.

But here’s the Efficient Markets theme. You, the next day, read about it in the Wall Street Journal in the morning. You’re now 24 hours late, OK? So, you call your broker and say, maybe I should buy this stock, they’ve got this new breakthrough. Your broker might laugh at you, right? Because you’re 24 hours late. And what do you know, anyway, about pharmaceuticals? That’s where the Efficient Markets Hypothesis is true. You can’t expect to routinely profit from information that’s already out there. If you’re going to profit, you’ve got to come up with something faster, something that you can get faster.

I think this is somewhat what David Swensen was referring to yesterday [correction: last lecture], when he talked about different asset classes. Remember how he talked about comparing the top quartile and bottom quartile of investment managers, in terms of returns, or at different asset classes? Well, the managers weren’t able to beat the bond market very much. The top quartile wasn’t able to beat the stock market very much. But when you get to unusual assets, private equity, which is not traded on stock exchanges, or absolute return investments that he talked about–they’re unusual, smaller, rare investments that the public doesn’t have a lot of information about. And these guys can get ahead on those things.

So, part of what makes Swensen a success is picking his game. Even so, the stock market is not completely efficient, but it’s so much more efficient, because it’s so many people involved in it and watching it. So, maybe I should write here, because this is what we’re talking about here. The Efficient Markets Hypothesis. I’ll write it down. This is the name for this idea that was coined–or sometimes people say Efficient Markets Theory. They’re referring to what Conant and Gibson and other people had been talking about for a long time, and it was common knowledge. But the first person to use this term apparently was Harry Roberts, a professor at the University of Chicago. But he was made famous by Eugene Fama, who referred to it as Harry Roberts’s idea. Eugene Fama is maybe the best-known finance professor in the country, I think. He is also at the University of Chicago. He’s been talked about as a Nobel Prize candidate for a long time, and he should have won probably, even though his theory is not entirely right. I think his chances of getting it have dimmed a little bit, because the theory is not looked upon as quite such an absolute truth as it used to be. As I said, it’s a half-truth.

What Conant said is all well taken and right, but there’s other nuances and it’s not exactly–when you first read Conant, you think, the guy is brilliant. That’s what I thought, this is right on. Then you read it again, you think, well, you know maybe there is a little gambling in the financial market. Things don’t always work right. So, he was maybe a little bit too positive.

Chapter 3. Testing the Efficient Markets Hypothesis [00:29:10]

I can give a little history of this. It was in 1960. The University of Chicago is kind of the forerunner in this. In 1960, Ford Foundation gave a grant to the University of Chicago to assemble all stock price data back to 1926. And to get it right, OK? So, they set up the Center for Research in Securities Prices at Chicago. The Center for Research in Securities Prices, or CRSP, as it’s called, had a Ford Foundation grant to go to the stock exchanges in the United States and get all the data, and get it right.

Remember, I told you they have splits in stocks. So, you’ll see the price of a share, and then suddenly the price will fall in half or maybe it will double, because they changed the units of measurement. If you wanted to know, what is the price history of stocks over the long haul, no one had ever organized that and figured out things like that, and got it right. And when were the dividends paid? When did you actually get the dividends? So they said, let’s get it right. Let’s put it on–hey, this is really super modern–a UNIVAC tape, OK? That’s a computer tape, all right? We’ll put it on a tape and we’ll sell it at cost to anybody in the world. And so the Ford Foundation, which is a non-profit, said: Good idea, let’s do it. So, that CRSP tape has launched a revolution in finance, because nobody had the data. They were throwing it away, it was not used. How do you know what Sharpe ratios are if you don’t have the data? I mean, you could find it in newspapers, but it wasn’t organized right, it wasn’t set up right.

So, with the invention of the CRSP tape in 1960, it really gave impetus to the Efficient Markets revolution. And by the end of that decade, there were thousands of articles testing market efficiency using the CRSP tape. And, in particular in 1969, Eugene Fama wrote one of the most cited articles in the history of finance. It was called “Efficient Capital Markets: A Review”. So, he reviews all of these studies of the CRSP tape. And it looked authoritative for the first time, because we were using the whole universe of stocks, all the way back to 1926. That sounded like a long time, and a lot of data. And Fama said it’s not uniform, there are some negative results, but the evidence is that markets are remarkably efficient. And this is a truth that we’ve discovered. And that, it was really a bombshell, because he was discrediting, or seeming to discredit, practically all the investment managers in the country. It was a huge industry. And he’s claiming the successful ones must have just been lucky, because the market is so efficient that we can’t see any way that you could make money in the market.

The high point of the Efficient Markets Hypothesis was probably in the 1970s. I’ll call that the high point. It seemed, at that time, that all of the scholars were finding that there was no way to beat the market. But it started to deteriorate, the support for the Efficient Markets Hypothesis. There’s been a change in thinking. Efficient Markets is still regarded with respect, but not the same respect that it had in 1969 or in 1979. I have here some indication of how thinking has changed about Efficient Markets. The textbook that I used to use for this course–I’ve been teaching this course for 25 years–Fabozzi et al. wasn’t even written when I started, so I was using a textbook called Brealey & Myers, Principles of Corporate Finance. I still have all these old editions. It’s a very successful textbook. Since I was teaching out of it all those years, I went back and looked–I don’t have the first edition of that book, I have the second edition, 1984–and I was teaching out of it in this same class in 1984.

At the end of that book, there’s a chapter on the seven most important ideas in finance. And one of the ideas is Efficient Markets. And quoting the textbook, Brealey and Myers say: “Security prices accurately reflect available information and respond rapidly to new information as soon as it becomes available.” They do qualify it, in 1984. “Don’t misunderstand the Efficient Markets idea. It doesn’t say there are no taxes or costs. It doesn’t say there aren’t some clever people and some stupid ones; it merely implies that competition in actual capital markets is very tough. There are no money machines and security prices reflect the true underlying value of assets.” OK? Let me repeat that: “Security prices reflect the true underlying value of assets.” OK? That’s a pretty strong statement, right? But that’s Efficient Markets. They just said it in their second edition of the book.

Not many people would say that, right? Trust the stock market, don’t trust people you know and love and trust. Trust the stock market. Well, they deleted that from later editions of their book. I think that’s a sign of changes. So, I was looking at the 2008 edition. They’ve now taken on a third author. They’re getting tired of coming out with more and more editions of their book, so they’ve taken on Franklin Allen from the Wharton School. They’ve deleted what I just read, and now it says–I’m quoting from them: “Much more research is needed before we have a full understanding of why asset prices sometimes get so out of line with what appears to be their discounted future payoffs.” That’s a complete turnaround in the textbook. This is one of the most popular textbooks, and they’ve changed completely. So, I think we have an idea that started around the 1960s. It was somehow associated with computers, and electronic databases, and modern thinking, and mathematical finance. They kind of went too far with it. They concluded that you just can’t beat the market.

Another thing I put on the reading list is a reading from The New Yorker magazine. It just came out in December. I thought it was relevant. It’s by Jonah Lehrer and the title of the article is “The Truth Wears Off”, subtitle, “Is There Something Wrong With the Scientific Method?” I don’t know there’s anything wrong with the scientific method, but I was interested in this article, because what The New Yorker article points out is that a lot of scientists–and this is outside of finance, I’m making a parallel here–a lot of scientists who follow careful scientific procedures seem to generate results that are later discredited. And nobody can figure out why. It’s like the universe is changing.

He gives an example in The New Yorker article–and this is from drugs–there’s a class of drugs called second-generation antipsychotics. These are used for people who are either schizophrenic or–I guess it can be used more generally than that–some of the drugs are called Abilify, Seroquel, Zyprexa. When these drugs were first introduced, careful studies that passed muster in the best medical journals found that they were highly effective. And they were written up as a godsend, a way of dealing with problems that used to weigh on people. Wonderful. The medical procedures involved careful controls on studies including a double blind procedure. When you want to test a drug on human subjects, both the subject doesn’t know whether he or she is getting the drug, and the experimenter, who runs experiment, doesn’t know which one is the drug. So, you give bottle A and bottle B to the experimenter, and the experimenter is never told, which one is Zyprexa and which one is a placebo. And then the experimenter has to write up a whole report on drug A and drug B, not even knowing, OK? This is to eliminate any possible bias. So, these drugs passed that. You see what I’m saying? The controls were right, everything was good.

And as years go by, the tests start coming out–the new attempts to replicate those start coming out more negative. They didn’t disprove the drugs; they just weren’t such wonder drugs as they thought. So, how can that be? And what the article says, well, it must be that somehow scientific bias, when there’s an enthusiasm for some new theory, it creeps in even if you try to make the strongest controls. You say, how could that happen with a double blind procedure? Well, maybe they broke the double blind somehow. They tried, but the guy, experimenter, figured it out. And then he started not deliberately fabricating results, but it’s the kind of thing where one subject says, I didn’t take my Abilify regularly. I took two tablets last week. I have to decide whether to throw this person out of the sample, and then I kind of remember that the drug wasn’t working for this person, and it colors my judgment, so I throw them out.

And another thing that happens is that the studies that didn’t find it might have been suppressed, right? Someone might have done an Abilify test and gotten bad results. And then showed it to their superior and said, should I publish this? And the superior said, wait a minute, there must be something wrong here. Abilify is wonderful, so let’s look. And then they find something that might be wrong, and he says, you shouldn’t publish this, because they find something. So, the publication process is biased for a while, but eventually it catches up. So, I think the same thing happened with the Efficient Markets Hypothesis. In the initial enthusiasm, anybody who found that the Efficient Markets Hypothesis wasn’t supported by the evidence, that person would be told, look again. Maybe you’ve done something wrong. So that’s what happened.

Chapter 4. Technical Analysis and the Head and Shoulders Pattern [00:40:49]

I wanted to do a little bit more history and describe the concept of Random Walk, which is central to the Efficient Markets Hypothesis. Let me start, though, with a little bit more history. Technical analysis. This term goes back–must be over 100 years. Technical analysis is the analysis of stock prices, or maybe other speculative asset prices, by looking at charts of the prices and looking for patterns that suggest movements in prices. The classic text of technical analysis is Edwards and McGee. McGee, there’s a famous story about him. He was not a professor; he was a Wall Street analyst. And the story about him is that he believed that you look at the prices and you can predict prices. And in fact, he said, I don’t want to look at anything else. I just want to see prices. I’ll do plots, and I can–using my judgment, I can figure out what it’s going to do. And so, the story about McGee is, everyone on Wall Street wants the corner office overlooking the World Trade Center, whatever. He said, I wanted an interior office with no windows. I don’t want any distractions; I don’t want the real world impinging on my judgment. And so, that’s McGee.

But he said that there are certain things that you see obviously. For example, resistance level. When the Dow Jones Industrial Average approached 1,000–I think it was in the 1960s–it’s way above that now, as you know. But when it first approached 1,000, technical analysts said, you know maybe it’s going to have trouble crossing 1,000, because that’s a psychological barrier. That’s sounds magical, how can the Dow be worth over 1,000? Wow, I’m going to sell. And so, the idea was that people would sell when it approached 1,000. And the technical analysts seemed to be right, because the Dow bounced around just below 1,000 for a long time. I guess it was months or a year, like it couldn’t cross the resistance level. That’s one example.

I have another example, which is from Edwards and McGee’s book.

[SIDE CONVERSATION]

Professor Robert Shiller: That’s Edwards and McGee; this is one of the patterns. McGee was actually a student of psychology, and he thought certain kinds of patterns seemed to have really spooked people. And this is one pattern, which he called ”Head and Shoulders.” That’s the head, that’s one shoulder, that’s the other shoulder. He said, when you see this pattern, watch out. It’s going to, actually, totally collapse, as it is shown doing. These are stock prices plotted against time. These are days, each of these points is a day. And this is from their book, so it’s a hypothetical. You hardly ever see such perfect Head and Shoulders patterns. And so, maybe that’s Edwards and McGee’s most–Head and Shoulders.

So, the question is, does it work? Does it really work? In the early 1970s, when the Efficient Markets Hypothesis was really strong, Burton Malkiel, who was a professor at Princeton, and then later he was the Dean of the Yale School of Management, wrote a book called A Random Walk Down Wall Street, which claimed that technical analysis was bunk. And he said, many studies have shown that it doesn’t work. This Head and Shoulders doesn’t work. None of Edwards and McGee’s stuff worked. There were lots of studies, but I actually met him at a cocktail party after his book came out. And I said, you didn’t footnote all those studies about technical analysis. Where are they? I can’t find them. I did a search. Not on the Internet, I did it on something else, but I was able to search. I couldn’t find them, where are they? And I found that he didn’t have an immediate answer.

I suspect that he was extrapolating–there was a literature on testing market efficiency. They looked for things like momentum, whether that continued. But there was something a little bit wrong with the literature. Not many people really confronted technical analysis. Later, there were people who did look at some of Edwards and McGee’s points, and they found some element of truth to them. So, I think the answer is, McGee wasn’t a total idiot, as you might infer from the Efficient Markets Theory. But it’s not going to make you rich, either. If anything, technical analysis is a subtle art that can augment trading strategies. I bet David Swensen doesn’t do it at all. I could have asked, I don’t know for sure.

Chapter 5. Random Walk vs. First-Order Autoregressive Process as Stock Price Model [00:47:04]

Let me talk about Random Walk, which is a central idea in finance. The idea is that, if stock prices are really efficient, then any change from day-to-day has to be due only to news. And news is essentially unforecastable. Therefore, stock prices have to do a Random Walk through time. That means that any future movement in them is always unpredictable. The changes are totally random. So, the term Random Walk is an important term. It was coined not by a finance theorist, but by a statistician, Karl Pearson, writing in the scientific journal Nature in 1905. Now, he didn’t link it to finance. But what he said is, the movements in some–well, he was thinking theoretically. Actually, I believe he used the example of a drunk. Let’s take someone who is so drunk that each step is random. This person has no direction at all, staggering randomly, OK? So, he starts out at a lamp pole. And what would you predict–this is what Pearson asked–what would you predict is his position in 10 minutes? He happens to be at a lamp pole right now. And what Pearson said is, well, your best forecast is that he’s right where he is now. Because you have no bias. He could go in any direction, equally likely. So, what’s most likely? It’s that he stays right where he is.

And what is the probability distribution? Well, it turns out that the standard deviation around that point goes up with √n steps, OK? Because each step is independent of the other, so the square root rule applies. So, if you’re asked to forecast his position after an hour–that’s a lot of steps–you would say, I predict he’s right where he is now, but I now have a big standard deviation around it. Pearson’s article is a very simple idea. Among the readers, apparently, was Albert Einstein and then Norbert Wiener, the mathematician who had invented a continuous version of the Random Walk, called the Wiener Process.

But it got into finance later. And in the Efficient Markets revolution, they started to realize that stock prices look more like more like a Random Walk than a Head and Shoulders. You look at these Head and Shoulders patterns and they’re hard to find. So, what is a Random Walk? Let me just define it. A Random Walk is where you have a series

where ε_t is noise. Just unforecastable noise: mean 0 and some standard deviation, OK? Ideally, it would be normally distributed, so it would have a bell-shaped curve, and then the math would be very easy and very simple.

So, I want I want to contrast that with an alternative, which is called a ”First-Order Autoregressive.” Let me get my notation here. Let’s take a process that starts at 100–that’s like the lamppost. We’ll say

So, this is an AR-1. That’s First-Order Autoregressive. It’s like a regression model where 100(1-ρ) is the constant term. And the coefficient of the lagged x is ρ. And we usually require that ρ is between -1 and 1. Normally, ρ is positive. So, that means that it’s mean reverting, but slowly. First of all, in the special case where ρ=1, I wouldn’t call it an AR-1 anymore, because it reduces to a Random Walk, right? If you make ρ 1, then the constant term drops out. I’ve got 100 minus 100–there’s no constant term–and then I’ve got

that’s a Random Walk, right?

So, in the extreme case where ρ gets to one, then a First-Order Autoregressive process converges to a Random Walk. I wanted to show you some simulations of it.

[SIDE CONVERSATION]

Professor Robert Shiller: OK, this here is a plot I had. Let’s first look at the black line. The black line is the Standard & Poor’s composite stock price index in real terms. I have that from 1871 until recently. That’s just there for comparison, that is the actual stock market. The pink line is a Random Walk that I generated using this formula [correction: The pink line is a Random Walk with a time trend, to be explained below.], and a random number generator that generates random normal variables, OK? I started them out at the same level, but don’t they look kind of similar? If you look at the stock market without comparing it with a Random Walk, it looks like it has patterns in it. In fact, here’s a Head and Shoulders, right here. Bang, bang, bang. When is that? I think this is 1937. This is–I am not sure–1930, 1931? And this is just before the war. I’m not sure exactly. It’s a nice Head and Shoulders. Hey, Edwards and McGee are sort of right, right? It dropped a lot after that. You know, I can find Head and Shoulders up here, too, right? Maybe.

The black line is actual U.S. history, that’s the stock market. The pink line is a fake stock line generated with pure random noise. And the fact that it’s going up is just chance. I can actually use this program to generate other examples. This should change, let me see, make sure it’s working here. The black line is the same. I’m not going to change the black line, the black line is history. I just did a brand new Random Walk calculation using my random number generator, which is there on Excel. That looks pretty good, doesn’t it? I’m going to do more for you. Which one is the real stock market? I find that hard to tell, right?

The insight is that people get deceived when they look at stock price charts. They think they see patterns. That pink line is guaranteed to have no patterns, because I generated it, so that there are no patterns, except random patterns. But when I look at this pink line, which just came up, look at that up-trend. Wow. That’s called a bull market, when it goes up. And I can make all kinds of theories about why that’s happening, but you know those would be fake theories, because I know what’s really happening. This is pure randomness. I’ll do it again, I can do this forever. That’s another one. Here the trend wasn’t quite so positive.

Oh, actually, I have to say one thing, I forgot. I did put an up-trend in the Random Walk. I am sorry. In this simulation, I did add a constant, so that it pushed it up. Otherwise, it was a Random Walk. I forgot what I did. It was a Random Walk with trend. But it doesn’t guarantee that it will go up, because it’s random. I’ll do another simulation. See, how fast I can do these? It’s the wonder of modern computers. If this were our history, people would say, the amazing stock market of the first half–look at that stock market in the first half of the 20th century! We would be devising all kinds of theories to explain it, but in fact it’s just nonsense. It’s just randomness. I’ll do a few more. Look at that. Boy, that would be a hump shape for the whole 20th century. We’d have historians trying to figure that one out. Oh, this one downturned. This is a bad outcome. Jeremy Siegel would not be pleased with this outcome, because it has the stock market gaining nothing in 100 years. These are all equally likely outcomes. Look at that one. If that was the world that we inherited, we would really think there was a linear trend in the market, right? People would be making models, saying–look at how straight that line is.

The point is that you start to see a sort of reality which is really just randomness. That’s why Nassim Taleb, a friend of mine, wrote a book called Fooled by Randomness. I thought that was a great title for a book and a great book. People don’t understand how things are just purely random, and your mind tries to make sense out of them. And you start looking at patterns and the patterns don’t mean anything. So, I can just keep doing this, but maybe I’ll stop. Look at that one. Of course, I’m helped along by the trend that I added in. So, I want to see if I can get a downtrend one. Because I put a trend in, it’s hard to get a real downtrend. That’s sort of a downtrend. That’s where there were a lot of negative shocks.

So, you see the comparison of the Random Walk with the actual stock market. So, the actual stock market looks a lot like a Random Walk. One thing is different, though, you don’t–look at this pattern, here. 1929, and this is the crash, after ‘29. You know, I’m not getting that in any of my simulations, and you know why I’m not? Because I chose a normally distributed shock. No fat tails, I didn’t put fat tails into my simulation. And you didn’t notice that, right? But in this simulation, we never see such sudden drops. So, there’s something that, if you spend time searching on it, you might see something not quite right. But basically, the Random Walk looks a lot like the actual stock market. I’m trying to get one that really matches up, but I’m not quite succeeding. That’s pretty good, isn’t it? In this simulation, 1929 wasn’t quite as strong and the Depression wasn’t as bad.

Anyway, what I want to do now is compare the Random Walk with the AR-1. Now I’m going to do the same thing, but I’m going to do it with –

[SIDE CONVERSATION]

Professor Robert Shiller: I know what I did, sorry. The pink line is now an AR-1 process, which is this process. So, it’s mean reverting now. I’m not comparing the Random Walk with the AR-1, I’m just doing the same thing now with an AR-1. Now, the thing about AR-1 is, you realize that it wants to come back to 100. I put a trend in. So, it’s actually coming back to a linear up-trend. What I did is I put in a time trend as well. But the point is that it tends to hug the trend somewhat. I have it here shown not around a trend, but around 100. What an AR-1 does is, say ρ is 1/2. If ρ is 1/2, then it means that if x_t-1 was above 100–last period it was above 100–it will be above 100 this time, but only half as much above 100, so it’s going back to 100. And then the next time, it’ll only be half, again, as much above 100 as it was the last time. So, it tends to go back to a trend.

But what I’ve shown here is a simulation with a random number generator of an AR-1 around a trend, where the trend matches the actual trend in the stock market. Now, in this case, this is not Random Walk. And there is a profit opportunity, and the profit opportunity is when it’s below trend, buy, when it’s above trend, sell. Because it will tend to come back to trend. I chose a ρ, which was very small, something like 1/2, I think, so it tends to come rapidly back to trend. This does look different than the actual stock market, doesn’t it? You see how much it hugs the trend? So, it doesn’t seem to fit as well. I can do simulations of this, too.

This is different. This is a different world. You can see the difference, right? This pink line doesn’t look as much like the actual stock market, because it really wants this trend. We saw trendy ones occasionally, by chance with a Random Walk, but here we’re seeing it’s always on a trend. And so, in this world, if the stock market were an AR-1, there would be a profitable strategy. Always buy when it’s below trend, and sell when it’s above trend. Because you know it’ll come back, you see how reliable this comes back to a trend?

So, you can see that there’s a fundamental difference, the Random Walk seems to fit the data better than the AR-1. The Random Walk theory says that stock prices are not mean reverting. Where they go from today is all random. If they’re above the historical trend–meaningless. The historical trend is just nonsense. It’s just random. And forget trends, forget anything. It’s always the drunk at the lamppost, no matter where you are in history. But this one, if ρ is substantially less than 1, it looks a lot different doesn’t it? With ρ = 1/2, this is not the world we live in. It would be too easy to make money. All these little oscillations around the trend I could profit from. But in the real world, it’s not like that.

But, what about, suppose the real world is AR-1 with ρ equal to 0.99, OK? What about that? Well that’s not much different from a Random Walk, is it? It’s going to be hard to tell the difference. In the Efficient Markets Theory period, people were really excited about the Random Walk Hypothesis. That’s why Burton Malkiel’s book, A Random Walk Down Wall Street, which he came out with–I think it was in 1973, right after Fama. It became a huge bestseller, it sold over a million copies, because at that time people were thinking, this is exciting new wisdom. We’ve learned the stock market is a Random Walk. And there’s all kinds of implications for that.

The problem is – and I have to wrap up–that the Random Walk Hypothesis wasn’t exactly right. It’s sort of right, you’ve gotten some insights. But you know, maybe the real world is AR-1 with a ρ close to 1. And in that world, there are profit opportunities, but they take a long time to come. So, if the real world is AR-1 with ρ equal to 0.99 or 0.98, that means you can buy stocks when they’re below trend. But then you have to wait 10, 20 years for them to get back to trend. So, it’s like the drunk on the lamp pole. The drunk is standing next to a lamp pole, it was a Random Walk. But now we put in elastic band around the drunk’s ankle and tie it to the lamp pole and it pulls him back. Now, if we have a very loose elastic, this guy can wander for a long time, but will eventually be pulled back. You’d never know when. But if you have a tight elastic, then it would be obvious that the drunk is coming back.

The problem is that the real world seems, maybe, to be more like the drunk with the loose elastic. And so, it’s kind of unsatisfying. You can beat the market, but simple trading rules like Edwards and McGee are not powerful, short-run profit opportunities. In that sense, the Efficient Markets Hypothesis is right. So, don’t forget the Efficient Markets Hypothesis. I’ll repeat what I said at the beginning. It’s a half-truth, it’s half true. Remember that, but don’t put too much faith in it, either.

[end of transcript]

Back to Top