And I want to talk today about efficient markets, which is a
theory that is a half-truth, I will say.
Before I start, I wanted to just give a few thoughts about
David Swensen's lecture last period.
Let me say, first of all, the Efficient Markets Hypothesis
or the Efficient Markets Theory is a theory that
markets efficiently incorporate all public
information.
And that, therefore, you cannot beat the market,
because the market has all the information in it.
You think you're smarter than the market,
that you know something?
No, the market knows more than you do.
And you'll find out that the market wins every time.
That's the Efficient Markets Hypothesis.
So, it's a very far-reaching hypothesis.
It means that, don't even try to beat the market.
That was a very rudimentary
introduction to today's lecture.
But here I brought in David Swensen, who is claimed to
have beaten the market consistently since 1985, and
dramatically.
And so, what do we make of that?
That's the subject of today's lecture.
By the way, after class one of you came up -- thank you, it
was nice, I don't know where you are --
one of you came up and thanked Swensen for his
scholarship at Yale.
Yale now has need-blind admissions for the world.
And so people, not just from the United States, people are
helped out, so that people who have managed to meet the high
admission standards here, it makes it possible for them to
actually come here.
And that's substantially David Swensen who did that, because
it's not just the generosity of the university.
They have to have the money to do it.
And so, somehow he seems to have made it.
And I know that there are still many cynics -- the
Efficient Markets Hypothesis has a lot of adherents still.
And as I say, it's a half-truth.
So, some people will say, well, Swensen was just lucky.
And I say, how could he have been lucky for
25 years in a row?
Well, not every single year, but pretty much.
And they say, well, you're picking the one guy out of
millions who is just the luckiest. So, those
arguments are made.
Anyway, one of you asked a question, which I thought it
was very good, at the end.
And that is why, in all of my discussion about Swensen and
all of his talk we never mentioned the Sharpe ratio?
Because as we said, the Sharpe ratio
corrects for risk taking.
That was one of our fundamental lessons.
And we showed you the Efficient Portfolio Frontier,
and the tangency line.
You can get any expected return you want at the expense
of higher uncertainty.
You do a very risky portfolio, and you have high expected
return, because of the risk.
If the risk is measured right.
But I think that's a good very good question.
I caught myself not correcting for it, I just said Yale's
portfolio had a high return.
I didn't correct for standard deviation of return.
So, David Swensen, in his answer, as you recall,
essentially said he doesn't believe in Sharpe ratios,
because we can't measure the standard deviation.
The Sharpe ratio is the excess return of a portfolio over the
market, divided by the standard
deviation of the return.
And that scales it down, so if the excess return is very high
but also has a very high standard deviation, that shows
they were just taking risks.
And so the Sharpe ratio would reveal that.
But Swensen said, I don't think that you can measure the
standard deviation of return.
Isn't that what is answer was?
You were here.
I may not be quoting him exactly right.
So, why wouldn't you be able to measure the standard
deviation of returns?
He gave a reason, which was that, well, when you're
looking at a broad portfolio like Yale's, a lot of the
things in there are private equity --
that means privately held, so it's not
traded on stock exchanges.
Or it's real estate.
Real estate is only traded every 10 years or 20 years,
and so who knows what it's worth?
All you have is an appraisal, but that's just some
appraiser's estimate, so the standard deviation would be
artificially low.
He's right about that, but I think there's
even more to that.
I know there's more to this.
There's a literature on this.
The point that I wanted to make is that you can do a --
suppose you're managing money.
And suppose the world out there is evaluating you by
your Sharpe ratio.
And suppose you have no ethics.
You just want money.
This isn't so obviously criminal, this is
not criminal I suppose.
So, you say, I just want to have the best Sharpe ratio for
a number of years running.
I'll get more and more people that will put money in my
investment fund, and eventually I
don't care what happens.
I'll move to Brazil or something.
Some foreign country -- pick out one.
I want to get out of here with the money.
So, all I have to do is fool people into thinking I have a
high Sharpe ratio for a while.
So, what do I do?
Well, there's an interesting paper on this by --
there's a lot of papers on this, but I'm
going to cite one --
by Professor Goetzmann and co-authors, here at Yale.
It's actually Goetzmann, Ibbotson, Spiegel, and Welch.
Maybe I'll put all their names on.
Roger Ibbotson is a professor here, Matt
Spiegel, and Ivo Welch.
What they did is they calculated the optimal
strategy for someone who wants to play games
with a Sharpe ratio.
So, you want to fool investors and get a spuriously high
Sharpe ratio.
And they found out what the optimal strategy is.
And that is to sell off the tails of your
distribution of returns.
So, if your return distribution looks like this
-- this is returns.
And you have a probability distribution, say a
bell-shaped curve.
And so the mean and standard deviation of this would be the
inputs to the Sharpe ratio.
But if you're cynical and you want to play tricks, what you
can do is sell the upper tail.
These are very unlikely good events.
Sell them and get money now, and then double up on the
lower tail.
So, you push the lower tail to something like that, and you
wipe out the upper tail, so it goes like that.
So, that means you'll get money, because you sold the
upper tail.
You would do that by selling calls -- we haven't talked
about options yet -- but you can do it by selling
out-of-the-money calls.
And you would do this by writing out-of-the-money puts.
OK, but what you do it you make it, so that if there's
really a bad year, it's going to be a doozer bad year for
your investors.
And if there's ever a good year, then, hey,
you won't get it.
But these good or bad years occur only infrequently.
So, in the meantime, you're making profits from these
sales and you have a high Sharpe ratio.
But little do they know --
you sold off the tails.
And so nothing happens for many years and you just look
like the best guy there.
So, it turns out that this is not just academic.
There was a company called Integral Investment Management
that did something like this strategy.
It was a hedge fund.
So, it was Integral Investment Management.
It did something like this, by trading in options.
And it got lots of investors to put millions in them.
Notably, the Art Institute of Chicago put $43 million into
this fund and its associated funds.
And then, in 2001, when the market dropped a lot, the Art
Institute of Chicago was wiped out.
They lost almost all of their $43 million.
And so, they got really angry, and they sued this company.
Because they said, you didn't tell us.
What have you been doing?
And then the company pointed out, in its defense, that it
actually said somewhere in the fine print that if markets go
down more than 30%, there would be a problem.
And somehow nobody at the Art Institute read that
or figured it out.
They thought the guy was a genius, because this company
had the highest Sharpe ratio in the industry.
You see what they're doing?
They're playing tricks.
They're making it look like there's less risk than
there really is.
And there's a strategy to do that.
But it didn't end well for Integral Investment
Management, because the Art Institute of Chicago managed
to stick them on other things -- they disclosed it.
They told people that they were doing this strategy.
The artists didn't figure it out.
But there were other dishonesties that they nailed
these guys on.
What Goetzmann and his co-authors did is they showed
that you can play tricks with the -- you can play a lot of
tricks in finance.
But one of them is to play a trick with the Sharpe ratio.
But their trick was very explicit.
It involved particular portfolio composition
involving options, and any professional would immediately
know that that's a trick.
And it didn't work for these guys, they were too aggressive
in their manipulation.
But you can do subtler things as a portfolio manager to get
your Sharpe ratio up.
Instead of manipulating with special derivatives positions
you could just buy companies that have large left tails.
They have a small probability of massive losses.
And you haven't done anything but pick a stock.
And nobody knows whether it really has a small probability
of massive losses, and you could
systematically invest in that.
And then you'd have a high Sharpe ratio for a while, and
then you'd just blow up and lose everything eventually.
I was thinking of an example from recent news.
What about the strategy of investing in Egyptian
companies that are tied to Mubarak?
It might have looked very good for a long time, right?
But the companies might have been underpriced, because
people sensed there's some instability in Egypt.
And look how fast it came on.
I don't know what the outcome will be at this point, but it
just happened -- bang.
That's a tail event, right?
You might look at Egyptian securities and think
everything is stable and fine, it's been 30 years, nothing
has happened.
But someone knows or suspects that there's
something maybe unstable.
And you, as an investor, wouldn't know that by looking
at the numbers.
What I'm getting at is really what is the essence of
Swensen's skill or contribution?
It's not his ability to manipulate Sharpe ratios or
numbers like that.
I think what it has, to me, in my mind, is something to do
with character and his real self and his real objectives.
And this gets at what people in finance are really doing.
I think that when you take a finance course in a
university, you may not get a proper appreciation of how
one, in a career in finance, develops a
reputation for integrity.
Nobody can really judge what you're doing as an investor,
because they can't judge it from the statistics.
Even though we've developed this nice theory about Sharpe
ratios and the like, you end up judging the person and what
the person's real objectives are.
I'll probably come back to that theme again.
Let me also say that's about doing the Goetzmann -- you
understand the strategy?
It's like investing in securities with
time bombs in them.
They're going to go off eventually, you don't know
exactly when.
And they look good for a while, but they'll blow up.
What does the law say about this?
Well, the law in the United States and other countries
emphasizes that an investment manager must not fail to
disclose relevant information about a security.
And it has to be more than boilerplate disclosure.
For, say Integral Investment Management, you could write up
a prospectus and say, but of course past returns are not a
guide to the future and something could go wrong.
That's a boilerplate disclosure, because people
think, well, that's what everybody says.
The law says that you have to actually actively disclose.
If there's something that's relevant that would make your
statistics misleading, you have to get their attention
and explain it to them.
That's the law.
So, I think it's laws like that, and it's the
development of --
it's not something that we specialize in academia.
Well, we are, we're trying to develop character I suppose.
But it's not just Sharpe ratios.
And I think that tendencies to rely on numbers like this has
led to errors in the past.
So, let me go more directly into today's lecture.
It's about the Efficient Markets Hypothesis, which is,
to me, a fascinating theory, which is not completely true,
which makes it all the more interesting.
The first statement that I could find --
I'm interested in the history of thought --
so, the first statement of the efficient markets hypothesis
that I could find was in a book by George --
[SIDE CONVERSATION]
PROFESSOR ROBERT SHILLER: George Gibson, who wrote a
book in 1889 called The Stock Exchanges of London, Paris,
and New York.
I'm quoting George Gibson, 1889: "When shares become
publicly known in an open market, the value, which they
acquire there, may be regarded as the judgment of the best
intelligence concerning them."
He described the stock market as a kind of voting machine
where people vote.
If you think a share is worth more, you vote by buying it.
If you think it's worth less in the market, you sell it.
And everybody in the world can do that.
It's open to the public.
So, the smartest people get into it, and then they soon
make a lot of money doing it, so they have a lot of votes.
So, the smarter people have more votes.
And you've got everyone there.
If anyone has a special clue, they go right in and they buy
if it's positive, or they sell if it's negative.
So, the smartest people go right in there and
aggressively affect the value.
Until it's right, and then there's no there's no
incentive to buy or sell.
The other thing about Gibson --
I should have copied the quote, but as I remember from
the book, he says something like ''in our modern electric
age, information flows with the speed of light.'' I say,
what is he talking about in 1889?
Well, you know what he's talking about.
The telegraph.
In fact, they had ticker machines.
They were electronic printers that would
print out stock quotes.
So, they were really in the information age by 1889.
So, this is what happened, Gibson said, you can't beat
the market.
It's just smarter because -- it's like Wikipedia is smarter
than any one of us, right?
Because it puts together all the
thinking of all the people.
Well, they had Wikipedia of a sort, because they had the
stock market.
They had the price.
So, that is the statement that --
he didn't use the word ''Efficient Markets.''
Actually, tried to find the origin of "Efficient Markets."
Sometimes in the 19th century, people would say ''Efficient
Markets.'' But it wasn't a cliche yet, it wasn't a phrase
that would be recognizable.
Even in this context, they would use it.
But it was not a name for a theory yet.
The next Efficient Markets theorist -- and I put this on
your reading list --
Charles Conant, who wrote a 1904 book called Wall Street
and the Country.
I put one chapter on that on the reading list, because it
was a statement of the Efficient Markets Hypothesis,
which was remarkably well-written, I thought.
He starts out the chapter by pointing out that a lot of
people think speculation is a kind of gambling,
or a kind of evil.
Speculating on the stock market, that sounds like some
wild activity that ought to be ruled out.
Then he said, how can that possibly be true?
The stock market is a central institution
of all modern economies.
To think that it's just gambling just
defies common sense.
Then he goes on and describes what it is that it does.
And there's some really nice passages in Conant's book.
In one passage he says, suppose for a moment that
stock markets of the world were
closed, what would happen?
He said no one would know what anything is worth.
No one could make any calculated decisions.
He said that, in fact, capital moves around from one industry
to another in respect to the prices that are quoted in
these markets.
And if you didn't see the prices, you would be blind.
It's often said about the Soviet economy --
which did not have stock markets or
financial markets --
that they relied on prices in the rest of the world.
Nobody in the Soviet Union could plan very well, because
they didn't know what anything was worth.
But at least they had the rest of the world, and they
thought, well that's an approximation to what prices
ought to be in the Soviet Union.
I just wanted to reiterate a little bit about the intuition
of Efficient Markets.
The idea is that if you trade securities, the advantage to
being there a little bit ahead of anyone else is enormous.
If you know five minutes before the other investors
about some good news or bad news, either way it doesn't
matter -- you know it five minutes earlier, you jump
right in and trade.
You can trade ahead of them and prices haven't changed
yet, you make money.
So, that has created an industry that speeds
information.
The first such industry that I would tell
you about is Reuters.
Mr. Reuters, I think in the 1840s before the telegraph,
created a financial information service using
carrier pigeons.
You know, these are birds.
So, when he was in London with some information, they had
carrier pigeons that were brought from Paris, and as
soon as new information came out, they would tie it to the
foot of the little bird and they'd let it go, and it would
fly to Paris.
It would go to its roosting place, the message would be
read, and subscribers would be notified.
And that was no joke, that really worked, because you
would have information hours or even days before everyone
else in Paris.
So, you could make a killing.
So, Reuters today, it's now called Thomson Reuters, is
still in that business.
The carrier pigeons were a brilliant idea.
But they had to keep up with the times, because, shortly
thereafter, the telegraph was invented and pigeons no longer
were the leading technology.
But maybe that was the beginning of the information
age, with pigeons.
Now we have beepers, and we have the internet.
A beeper is something you can carry in your pocket that
beeps when there's financial news.
So, suppose a company makes an announcement that that it has,
say, a new drug that's successful in trials.
As soon as they make the announcement, it then goes out
electronically everywhere, and the beepers start beeping.
And all the investment analysts, they drop their
morning coffee, because they know they have to act fast.
So, you get this new announcement.
Now it's t plus 20 seconds.
He's got his drug specialist on the phone.
What does this mean?
Quick, how much is it going to go up?
And so the guy says, I don't know, first thought, maybe
it's going to go up $2 a share.
OK, it's only gone up $1 a share, I'll buy right now.
And now it's two minutes after the announcement, and then the
analyst says, I've thought about it a little bit more.
No, I only think it's $1.50 a share.
This is homing in, and so the price is jiggling around
rapidly as all this is happening for a few minutes,
and then it settles down.
Because after --
I may be exaggerating -- after 10 minutes, they've kind of
figured it out and it's reached its new level.
They'll be thinking about it the next morning when they're
taking their shower, and they'll get a better and
better, more refined idea of what the price is.
But here's the Efficient Markets theme.
You, the next day, read about it in the Wall Street Journal
in the morning.
You're now 24 hours late.
So, you call your broker and say, maybe I should buy this
stock, they've got this new breakthrough.
Your broker might laugh at you, right?
Because you're 24 hours late.
And what do you know, anyway, about pharmaceuticals?
That's where the Efficient Markets Hypothesis is true.
You can't expect to routinely profit from information that's
already out there.
If you're going to profit, you've got to come up with
something faster, something that you can get faster.
I think this is somewhat what David Swensen was referring to
yesterday [correction: last lecture],
when he talked about different asset classes.
Remember how he talked about comparing the top quartile and
bottom quartile of investment managers, in terms of returns,
or at different asset classes?
Well, the managers weren't able to beat the
bond market very much.
The top quartile wasn't able to beat the
stock market very much.
But when you get to unusual assets, private equity, which
is not traded on stock exchanges, or absolute return
investments that he talked about -- they're unusual,
smaller, rare investments that the public doesn't have a lot
of information about.
And these guys can get ahead on those things.
So, part of what makes Swensen a success is picking his game.
Even so the stock market is not completely efficient, but
it's so much more efficient, because it's so many people
involved in it and watching it.
So, maybe I should write here, because this is what we're
talking about here.
The Efficient Markets Hypothesis.
I'll write it down.
This is the name for this idea that was coined --
or sometimes people say Efficient Markets theory.
They're referring to what Conant and Gibson and other
people had been talking about for a long time, and it was
common knowledge.
But the first person to use this term apparently was Harry
Roberts, a professor at the University of Chicago.
But he was made famous by Eugene Fama, who referred to
it as Harry Roberts's idea.
Eugene Fama is maybe the best-known finance professor
in the country, I think.
He is also at the University of Chicago.
He's been talked about as a Nobel Prize candidate for a
long time, and he should have won probably, even though his
theory is not entirely right.
I think his chances of getting it have dimmed a little bit,
because the theory is not looked upon as quite such an
absolute truth as it used to be.
As I said, it's a half-truth.
What Conant said is all well-taken and right, but
there's other nuances and it's not exactly --
when you first read Conant, you
think, the guy is brilliant.
That's what I thought, this is right on.
Then you read it again, you think, well, you know maybe
there is a little gambling in the financial market.
Things don't always work right.
So, he was maybe a little bit too positive.
I can give a little history of this.
It was in 1960.
The University of Chicago is kind of the
forerunner in this.
In 1960, Ford Foundation gave a grant to the University of
Chicago to assemble all stock price data back to 1926.
And to get it right.
So, they set up the Center for Research in Securities Prices
at Chicago.
The Center for Research in Securities Prices, or CRSP, as
it's called, had a Ford Foundation grant to go to the
stock exchanges in the United States and get all the data,
and get it right.
Remember, I told you they have splits in stocks.
So, you see the price of a share, and then suddenly the
price will fall in half or maybe it will double, because
they changed the units of measurement.
If you wanted to know, what is the price history of stocks
over the long haul, no one had ever organized that and
figured out things like that, and got it right.
And when were the dividends paid?
When did you actually get the dividends?
So they said, let's get it right.
Let's put it on -- hey, this is really super modern -- a
UNIVAC tape.
That's a computer tape.
We'll put it on a tape and we'll sell it at cost to
anybody in the world.
And so the Ford Foundation, which is a non-profit, said:
Good idea, let's do it.
So, that CRSP tape has launched a revolution in
finance, because nobody had the data.
They were throwing it away, it was not used.
How do you know what Sharpe ratios are if you
don't have the data?
I mean, you could find it in newspapers, but it wasn't
organized right, it wasn't set up right.
So, with the invention of the CRSP tape in 1960, it really
gave impetus to the Efficient Markets revolution.
And by the end of that decade, there were thousands of
articles testing market efficiency
using the CRSP tape.
And, in particular in 1969, Eugene Fama wrote one of the
most cited articles in the history of finance.
It was called "Efficient Capital Markets: A Review".
So, he reviews all of these studies of the CRSP tape.
And it looked authoritative for the first time, because we
were using the whole universe of stocks, all
the way back to 1926.
That sounded like a long time, and a lot of data.
And Fama said it's not uniform, there are some
negative results, but the evidence is that markets are
remarkably efficient.
And this is a truth that we've discovered.
And that was really a bombshell, because he was
discrediting, or seeming to discredit, practically all the
investment managers in the country.
It was a huge industry.
And he's claiming the successful ones must have just
been lucky, because the market is so efficient that we can't
see any way that you could make money in the market.
The high point of the Efficient Markets Hypothesis
was probably in the 1970s.
I'll call that the high point.
It seemed, at that time, that all of the scholars were
finding that there was no way to beat the market.
But it started to deteriorate, the support for the Efficient
Markets Hypothesis.
There's been a change in thinking.
Efficient Markets is still regarded with respect, but not
the same respect that it had in 1969 or in 1979.
I have here some indication of how thinking has changed about
Efficient Markets.
The textbook that I used to use for this course --
I've been teaching this course for 25 years --
Fabozzi et al.
wasn't even written when I started, so I was using a
textbook called Brealey & Myers, Principles
of Corporate Finance.
I still have all these old editions.
It's a very successful textbook.
Since I was teaching out of it all those years, I went back
and looked --
I don't have the first edition of that book, I have the
second edition, 1984 -- and I was teaching out of it in this
same class in 1984.
At the end of that book, there's a chapter on the seven
most important ideas in finance.
And one of the ideas is Efficient Markets.
And quoting the textbook, Brealey and Myers say:
"Security prices accurately reflect available information
and respond rapidly to new information as soon as it
becomes available." They they do qualify it, in 1984.
"Don't misunderstand the Efficient Markets idea.
It doesn't say there are no taxes or costs.
It doesn't say there aren't some clever people and some
stupid ones, it merely implies that competition in actual
capital markets is very tough.
There are no money machines and security prices reflect
the true underlying value of assets." Let me repeat that:
"Security prices reflect the true underlying value of
assets." That's a pretty strong statement, right?
But that's Efficient Markets.
They just said it in their second edition of the book.
Not many people would say that, right?
Trust the stock market, don't trust people you know and love
and trust. Trust the stock market.
Well, they deleted that from later editions of their book.
I think that's a sign of changes.
So, I was looking at the 2008 edition.
They've now taken on a third author.
They're getting tired of coming out with more and more
editions of their book, so they've taken on Franklin
Allen from the Wharton School.
They've deleted what I just read, and now it says --
I'm quoting from them: "Much more research is needed before
we have a full understanding of why asset prices sometimes
get so out of line with what appears to be their discounted
future payoffs." That's a complete
turnaround in the textbook.
This is one of the most popular textbooks, and they've
changed completely.
So, I think we have an idea that started around the 1960s.
It was somehow associated with computers, and electronic
databases, and modern thinking, and
mathematical finance.
They kind of went too far with it.
They concluded that you just can't beat the market.
Another thing I put on the reading list is a reading from
The New Yorker magazine.
It just came out in December.
I thought it was relevant.
It's by Jonah Lehrer and the title of the article is "The
Truth Wears Off: Is There Something Wrong With the
Scientific Method?" I don't think there's anything wrong
with the scientific method, but I was interested in this
article, because what The New Yorker article points out is
that a lot of scientists -- and this is outside of
finance, I'm making a parallel here -- a lot of scientists
who follow careful scientific procedures seem to generate
results that are later discredited.
And nobody can figure out why.
It's like the universe is changing.
He gives an example in The New Yorker article --
and this is from drugs --
there's a class of drugs called second-generation
antipsychotics.
These are used for people who are either schizophrenic or --
I guess it can be used more generally than that -- some of
the drugs are called Abilify, Seroquel, Zyprexa.
When these drugs were first introduced, careful studies
that passed muster in the best medical journals found that
they were highly effective.
And they were written up as a godsend, a way of dealing with
problems that used to weigh on people.
Wonderful.
The medical procedures involved careful controls on
studies including a double blind procedure.
When you want to test a drug on human subjects, both the
subject doesn't know whether he or she is getting the drug,
and the experimenter, who runs experiment, doesn't know which
one is the drug.
So, you give bottle A and bottle B to the experimenter,
and the experimenter is never told, which one is Zyprexa and
which one is a placebo.
And then the experimenter has to write up a whole report on
drug A and drug B, not even knowing.
This is to eliminate any possible bias.
So, these drugs passed that.
You see what I'm saying?
The controls were right, everything was good.
And as years go by, the tests start coming out -- the new
attempts to replicate those start
coming out more negative.
They didn't disprove the drugs, they just weren't such
wonder drugs as they thought.
So, how can that be?
And what the article says, well, it must be that somehow
scientific bias, when there's an enthusiasm for some new
theory, it creeps in even if you try to make
the strongest controls.
You say, how could that happen with a double blind procedure?
Well, maybe they broke the double blind somehow.
They tried, but the guy, experimenter, figured it out.
And then he started not deliberately fabricating
results, but it's the kind of thing where one subject says,
I didn't take my Abilify regularly.
I took two tablets last week.
I have to decide whether to throw this person out of the
sample, and then I kind of remember that the drug wasn't
working for this person, and it colors my judgment, so I
throw them out.
And another thing that happens is that the studies that
didn't find it might have been suppressed.
Someone might have done an Abilify test
and gotten bad results.
And then showed it to their superior and said, should I
publish this?
And the superior said, wait a minute, there must be
something wrong here.
Abilify is wonderful, so let's look.
And then they find something that might be wrong, and he
says, you shouldn't publish this,
because they find something.
So, the publication process is biased for a while, but
eventually it catches up.
So, I think the same thing happened with the Efficient
Markets Hypothesis.
In the initial enthusiasm, anybody who found that the
Efficient Markets Hypothesis wasn't supported by the
evidence, that person would be told, look again.
Maybe you've done something wrong.
So that's what happened.
I wanted to do a little bit more history and describe the
concept of Random Walk, which is central to the Efficient
Markets Hypothesis.
Let me start, though, with a little bit more history.
Technical Analysis.
This term goes back --
must be over 100 years.
Technical analysis is the analysis of stock prices, or
maybe other speculative asset prices, by looking at charts
of the prices and looking for patterns that suggest
movements in prices.
The classic text of technical analysis is Edwards and McGee.
McGee, there's a famous story about him.
He was not a professor, he was a Wall Street analyst. And the
story about him is that he believed that you look at the
prices and you can predict prices.
And in fact, he said, I don't want to look at anything else.
I just want to see prices.
I'll do plots, and I can --
using my judgment, I can figure out what
it's going to do.
And so, the story about McGee is, everyone on Wall Street
wants the corner office overlooking the World Trade
Center, whatever.
He said, I wanted an interior office with no windows.
I don't want any distractions, I don't want the real world
impinging in my judgment.
And so, that's McGee.
But he said that there are certain things
that you see obviously.
For example, resistance level.
When the Dow Jones Industrial Average approached 1,000 --
I think it was in the 1960s --
it's way above that now, as you know.
But when it first approached 1,000, technical analysts
said, you know maybe it's going to have trouble crossing
1,000, because that's a psychological barrier.
That's sounds magical, how can the Dow be worth over 1,000?
Wow, I'm going to sell.
And so, the idea was that people would sell when it
approached 1,000.
And the technical analysts seemed to be right, because
the Dow bounced around just below 1,000 for a long time.
I guess it was months or a year, like it couldn't cross
the resistance level.
That's one example.
I have another example, which is from
Edwards and McGee's book.
[SIDE CONVERSATION]
PROFESSOR ROBERT SHILLER: That's Edwards and McGee, this
is one of the patterns.
McGee was actually a student of psychology, and he thought
certain kinds of patterns seemed to have
really spooked people.
And this is one pattern, which he called ''Head and
Shoulders.'' That's the head, that's one shoulder, that's
the other shoulder.
He said, when you see this pattern, watch out.
It's going to, actually, totally collapse, as it is
shown doing.
These are stock prices plotted against time.
These are days, each of these points is a day.
And this is from their book, so it's hypothetical.
You hardly ever see such perfect Head
and Shoulders patterns.
And so, maybe that's Edwards and McGee's most famous.
Head and Shoulders.
So, the question is, does it work?
Does it really work?
In the early 1970s, when the efficient markets hypothesis
was really strong, Burton Malkiel, who was a professor
at Princeton, and then later he was the Dean of the Yale
School of Management, wrote a book called A Random Walk Down
Wall Street, which claimed that technical
analysis was bunk.
And he said many studies have shown that it doesn't work.
This Head and Shoulders doesn't work.
None of Edwards and McGee's stuff worked.
There were lots of studies, and I actually met him at a
cocktail party after his book came out.
And I said, you didn't footnote all those studies
about technical analysis.
Where are they?
I can't find them.
I did a search.
Not on the internet, I did it on something else, but I was
able to search.
I couldn't find them, where are they?
And I found that he didn't have an immediate answer.
I suspect that he was extrapolating -- there was a
literature on testing market efficiency.
They looked for things like momentum,
whether that continued.
But there was something a little bit wrong with the
literature.
Not many people really confronted technical analysis.
Later, there were people who did look at some of Edwards
and McGee's points, and they found some
element of truth to them.
So, I think the answer is, McGee wasn't a total idiot, as
you might infer from the Efficient Markets Theory.
But it's not going to make you rich, either.
If anything, technical analysis is a subtle art that
can augment trading strategies.
I bet David Swensen doesn't do it at all.
I could have asked, I don't know for sure.
Let me talk about Random Walk, which is a
central idea in finance.
The idea is that, if stock prices are really efficient,
then any change from day to day has to
be due only to news.
And news is essentially unforecastable.
Therefore, stock prices have to do a random
walk through time.
That means that any future movement in them is always
unpredictable.
The changes are totally random.
So, the term Random Walk is an important term.
It was coined not by a finance theorist, but by a
statistician, Karl Pearson, writing in the scientific
journal Nature in 1905.
Now, he didn't link it to finance.
But what he said is, the movements in some -- well, he
was thinking theoretically.
Actually, I believe he used the example of a drunk.
Let's take someone who is so drunk that
each step is random.
This person has no direction at all, staggering randomly.
So, he starts out at a lamp pole.
And what would you predict -- this is what Pearson asked --
what would you predict is his position in 10 minutes?
He happens to be at a lamp pole right now.
And what Pearson said is, well, your best forecast is
that he's right where he is now.
Because you have no bias.
He could go in any direction, equally likely.
So, what's most likely?
It's that he stays right where he is.
And what is the probability distribution?
Well, it turns out that the standard deviation around that
point goes up with the square root of n steps.
Because each step is independent of the other, so
the square root rule applies.
So, if you're asked to forecast his position after an
hour -- that's a lot of steps -- you would say, I predict
he's right where he is now, but I now have a big standard
deviation around it.
Pearson's article is a very simple idea.
Among the readers, apparently, was Albert Einstein and
Norbert Wiener, the mathematician who had invented
a continuous version of the Random Walk,
called the Wiener Process.
But it got into finance later.
And in the Efficient Markets revolution, they started to
realize that stock prices look more like more like a Random
Walk than a Head and Shoulders.
You look at these Head and Shoulders patterns and they're
hard to find.
So, what is a Random Walk?
Let me just define it.
A Random Walk is where you have a series x sub t equals x
sub t minus 1 plus epsilon sub t, where
epsilon sub t is noise.
Just unforecastable noise: mean 0 and
some standard deviation.
Ideally, it would be normally distributed, so it would have
a bell-shaped curve, and then the math would be very easy
and very simple.
So, I want I want to contrast that with an alternative,
which is called a ''First-Order
Autoregressive.'' Let me get my notation here.
Let's take a process that starts at 100 -- that's like
the lamp post. We'll say x sub t equals -- how am I
putting this --
100 plus some number rho -- that's a rho -- times x sub t
minus 1 minus 100 plus epsilon sub t.
So, this is an AR-1.
That's First-Order Autoregressive.
It's like a regression model where 100 times 1 minus rho is
the constant term.
And the coefficient of the lagged x is rho.
And we usually require that rho is between minus 1 and 1.
Normally, rho is positive.
So, that means that it's mean-reverting, but slowly.
First of all, in a special case where rho equals 1, I
wouldn't call it an AR-1 anymore, because it reduces to
a Random Walk, right?
If you make rho 1, then the constant term drops out.
I've got 100 minus 100 --
there's no constant term -- and then I've got x sub t
equals x sub t minus 1 plus epsilon sub t,
that's a Random Walk.
So, in the extreme case where rho gets to one, then a
First-Order Autoregressive process
converges to a Random Walk.
I wanted to show you some simulations of it.
[SIDE CONVERSATION]
PROFESSOR ROBERT SHILLER: OK, this here is a plot I had.
Let's first look at the black line.
The black line is the Standard & Poor's composite stock price
index in real terms. I have that from 1871 until recently.
That's just there for comparison, that is the actual
stock market.
The pink line is a Random Walk that I generated using this
formula [correction: The pink line is a Random Walk with a
time trend, to be explained below.], and a random number
generator that generates random normal variables.
I started them out at the same level, but don't they look
kind of similar?
If you look at the stock market without comparing it
with a Random Walk, it looks like it has patterns in it.
In fact, here's a Head and Shoulders, right here.
Bang, bang, bang.
When is that?
I think this is 1937.
This is --
I am not sure --
1930, 1931?
And this is just before the war.
I'm not sure exactly.
It's a nice Head and Shoulders.
Hey, Edwards and McGee are sort of right, right?
It dropped a lot after that.
You know, I can find Head and Shoulders up here, too, right?
Maybe.
The black line is actual U.S. history,
that's the stock market.
The pink line is a fake stock line generated with pure
random noise.
And the fact that it's going up is just chance.
I can actually use this program to
generate other examples.
This should change, let me see, make sure
it's working here.
The black line is the same.
I'm not going to change the black line, the
black line is history.
I just did a brand new Random Walk calculation using my
random number generator, which is there on Excel.
That looks pretty good, doesn't it?
I'm going to do more for you.
Which one is the real stock market?
I find that hard to tell, right?
The insight is that people get deceived when they look at
stock price charts.
They think they see patterns.
That pink line is guaranteed to have no patterns, because I
generated it, so that there are no patterns,
except random patterns.
But when I look at this pink line, which just came up, look
at that up-trend.
Wow.
That's called a bull market, when it goes up.
And I can make all kinds of theories about why that's
happening, but you know those would be fake theories,
because I know what's really happening.
This is pure randomness.
I'll do it again, I can do this forever.
That's another one.
Here the trend wasn't quite so positive.
Oh, actually, I have to say one thing, I forgot.
I did put an up-trend in the Random Walk.
I am sorry.
In this simulation, I did add a constant, so that
it pushed it up.
Otherwise, it was a Random Walk.
I forgot what I did.
It was a Random Walk with trend.
But it doesn't guarantee that it will go up,
because it's random.
I'll do another simulation.
See, how fast I can do these?
It's the wonder of modern computers.
If this were our history, people would say, the amazing
stock market of the first half -- look at that stock market
in the first half of the 20th century!
We would be devising all kinds of theories to explain it, but
in fact it's just nonsense.
It's just randomness.
I'll do a few more.
Look at that.
Boy, that would be a hump shape for
the whole 20th century.
We'd have historians trying to figure that one out.
Oh, this one downturned.
This is a bad outcome.
Jeremy Siegel would not be pleased with this outcome,
because it has the stock market gaining
nothing in 100 years.
These are all equally likely outcomes.
Look at that one.
If that was the world that we inherited, we would really
think there was a linear trend in the market, right?
People would be making models, saying --
look at how straight that line is.
The point is that you start to see a sort of reality which is
really just randomness.
That's why Nassim Taleb, a friend of mine, wrote a book
called Fooled by Randomness.
I thought that was a great title for a
book and a great book.
People don't understand how things are just purely random,
and your mind tries to make sense out of them.
And you start looking at patterns and the patterns
don't mean anything.
So, I can just keep doing this, but maybe I'll stop.
Look at that one.
Of course, I'm helped along by the trend that I added in.
So, I want to see if I can get a down-trend one.
Because I put a trend in, it's hard to get a real down-trend.
That's sort of a down-trend.
That's where there were a lot of negative shocks.
So, you see the comparison of the Random Walk with the
actual stock market.
So, the actual stock market looks a lot
like a random walk.
One thing is different, though, you don't -- look at
this pattern, here.
In 1929, and this is the crash, after '29.
You know, I'm not getting that in any of my simulations, and
you know why I'm not?
Because I chose a normally distributed shock.
No fat tails, I didn't put fat tails into my simulation.
And you didn't notice that, right?
But in this simulation, we never see such sudden drops.
So, there's something that, if you spend time searching on
it, you might see something not quite right.
But basically, the Random Walk looks a lot like the actual
stock market.
I'm trying to get one that really matches up, but I'm not
quite succeeding.
That's pretty good, isn't it?
In this simulation, 1929 wasn't quite as strong and the
Depression wasn't as bad.
Anyway, what I want to do now is compare the Random Walk
with the AR-1.
Now I'm going to do the same thing, but I'm going
to do it with --
[SIDE CONVERSATION]
PROFESSOR ROBERT SHILLER: I know what I did, sorry.
The pink line is now an AR-1 process,
which is this process.
So, it's mean reverting now.
I'm not comparing the Random Walk with the AR-1, I'm just
doing the same thing now with an AR-1.
Now, the thing about AR-1 is, you realize that it wants to
come back to 100.
I put a trend in.
So, it's actually coming back to a linear up-trend.
What I did is I put in a time trend as well.
But the point is that it tends to hug the trend somewhat.
I have it here shown not around a
trend, but around 100.
What an AR-1 does is, say rho is 1/2.
If rho is 1/2, then it means that if x sub t minus 1 was
above 100 --
last period it was above 100 --
it will be above 100 this time, but only half as much
above 100, so it's going back to 100.
And then the next time, it'll only be half, again, as much
above 100 as it was the last time.
So, it tends to go back to a trend.
But what I've shown here is a simulation with a random
number generator of an AR-1 around a trend, where the
trend matches the actual trend in the stock market.
Now, in this case, this is not Random Walk.
And there is a profit opportunity, and the profit
opportunity is when it's below trend, buy, when it's above
trend, sell.
Because it will tend to come back to trend.
I chose a rho which was very small, something like 1/2, so
it tends to come rapidly back to trend.
This does look different than the actual stock market,
doesn't it?
You see how much it hugs the trend?
So, it doesn't seem to fit as well.
I can do simulations of this, too.
This is different.
You can see the difference, right?
This pink line doesn't look as much like the actual stock
market, because it really wants this trend.
We saw trendy ones occasionally, by chance with a
Random Walk, but here we're seeing it's always on a trend.
And so, in this world, if the stock market were an AR-1,
there would be a profitable strategy.
Always buy when it's below trend, and sell when it's
above trend.
Because you know it'll come back, you see how reliable
this comes back to a trend?
So, you can see that there's a fundamental difference, the
Random Walk seems to fit the data better than the AR-1.
The Random Walk theory says that stock prices
are not mean reverting.
Where they go from today is all random.
If they're above the historical trend, it's
meaningless.
The historical trend is just nonsense.
It's just random.
And forget trends, forget anything.
It's always the drunk at the lamp post, no matter where you
are in history.
This one, if rho is substantially less than 1, it
looks a lot different doesn't it?
With rho equals 1/2, this is not the world we live in.
It would be too easy to make money.
All these little oscillations around the trend I could
profit from.
But in the real world, it's not like that.
But suppose the real world is AR-1 with rho equal to 0.99.
What about that?
Well that's not much different from a Random Walk, is it?
It's going to be hard to tell the difference.
In the Efficient Markets Theory period, people were
really excited about the Random Walk Hypothesis.
That's why Burton Malkiel's book, A Random Walk Down Wall
Street, which he came out with --
I think it was in 1973, right after Fama.
It became a huge best seller, it sold over a million copies,
because at that time people were thinking, this is
exciting new wisdom.
We've learned that the stock market is a Random Walk.
And there's all kinds of implications for that.
The problem is --
I have to wrap up --
that the Random Walk Hypothesis
wasn't exactly right.
It's sort of right, you've gotten some insights.
But you know, maybe the real world is AR-1 with a
rho close to 1.
And in that world, there are profit opportunities, but they
take a long time to come.
So, if the real world is AR-1 with rho equal to 0.99 or
0.98, that means you can buy stocks when
they're below trend.
But then you have to wait 10, 20 years for them
to get back to trend.
So, it's like the drunk on the lamp pole.
The drunk is standing next to a lamp pole, it
was a random walk.
But now we put in elastic band around the drunk's ankle and
tie it to the lamp pole and it pulls him back.
Now, if we have a very loose elastic, this guy can wander
for a long time, but will eventually be pulled back.
You'd never know when.
But if you have a tight elastic, then it would be
obvious that the drunk is coming back.
The problem is that the real world seems, maybe, to be more
like the drunk with the loose elastic.
And so, it's kind of unsatisfying.
You can beat the market, but simple trading rules like
Edwards and McGee are not powerful, short-run profit
opportunities.
In that sense, the Efficient Markets Hypothesis is right.
So, don't forget the Efficient Markets Hypothesis.
I'll repeat what I said at the beginning.
It's a half-truth, it's half true.
Remember that, but don't put too much faith in it, either.