Efficient Frontier
Efficient Frontiers Logo
William J. Bernstein

How Random a Walk?

Technical analysis is the Rodney Dangerfield of financial work—it gets no respect. The notion that one can divine the future price of a stock or index simply by looking at its price graph seems preposterous. The fastest way to bring a sneer to the face of most finance professionals and academics is to say the words "resistance level."

Pooh-poohing charts is equivalent to saying that security prices are a "random walk." To wit, there is no information contained in an asset’s prior price behavior: the fact that a security or market has just risen or fallen in price over a certain time period tells us nothing going forward.

I’ve been as guilty of this attitude as most; consider this gem from The Intelligent Asset Allocator:

In fact, there is an entire school of stock analysis which relies on the so called "relative strength" of a stock… the more rapidly it is rising, the better a buy it must be! Such idiocy boggles the mind.

Well, I didn’t get it quite right. It turns out that there is a pretty impressive literature demonstrating that asset prices are nonrandom, and that gazing at charts may not be a waste of time.

The Process

First of all, how exactly does one go about looking for nonrandom behavior? There are dozens of ways to do so, but the simplest is to look for "autocorrelations" in price changes. What we are in effect asking is, "Does the price change from the previous day/week/month/year/decade correlate with the price change of the succeeding day/week/month/year/decade?"

Let’s take the monthly returns for the S&P 500 from January 1926 to September 1998. That’s 873 months. What we now have is two series of 872 monthly returns, offset by one month. Thanks to the magic of modern spreadsheets, it is a simple matter to calculate a correlation coefficient of these two series. In other words, we are correlating each month’s return with the next. (To review, a correlation of +1 means that two series of data are perfectly correlated, 0 means that they are not correlated, and -1 that they are perfectly inversely correlated.)

It turns out that the autocorrelation of the monthly returns for 1926 through 1998 is 0.081. Not terribly impressive, but positive nonetheless, meaning that a good return this month means a slightly better than average chance of a good return next month. What are the odds that this could have happened by chance? In order to determine this, we have to calculate the standard deviation of autocorrelations for a data series of 873 random data points. The formula for this is sqrt(n-1)/n, which for 873 is 0.034. Thus, the autocorrelation of 0.081 is more than twice the "random walk" standard deviation of 0.034. This in turn means that the odds of this occurring with 873 random numbers is less than one in a hundred.

So, yes, US security prices exhibit some momentum over periods of one month.

An Unfair Advantage
You probably didn’t know this, but investors come in two shapes—convex and concave. Sharpe and Perold, in a classic piece in Financial Analysts Journal in 1985, defined the former as one who tends to buy when prices are rising, and the latter as one who buys when prices are falling: in other words, momentum players and contrarian investors. I suspect that the convex/concave dichotomy is a deeply behavioral phenomenon—you’re born either one or the other. The percentage of each who enjoy long walks in the park versus those who sky dive is probably radically different. The authors make the interesting point that in a market dominated by concave investors, it is better to be convex, and vice versa.

The two styles of investing are completely different—browse any investment discussion board and you’ll find that these two species tend to get on each other’s nerves quite easily. Efficient Frontier has a most definite concave bias towards buy-and-hold and rebalancing. This mandates buying when prices are falling.

The plain fact of the matter is, demonstrating that short-term momentum exists (or does not exist) is relatively easy, whereas demonstrating the same for long-term mean reversion is nearly impossible. Consider the 1926-98 period. Since 1926 there have been over 18,000 trading days, 3800 weeks, 873 months, 72 years, and 18 four-year periods. (Four years is the sort of time frame in which mean reversion of asset prices occurs.) Thus, while there is an abundance of data with which to look for short-term momentum, there is a distinct shortage of data with which to look for long-term mean reversion. If you toss 100 coins and come up with 55% heads, the result is most likely due to chance. But if you toss one million coins and come up with 55% heads, the coin is almost certainly loaded. This is because the standard deviation of percent heads tossed for 100 coin tosses is much larger than for one million.

In the case at hand, the large number of monthly data points with respect to momentum investing results in a standard deviation of autocorrelations of only 0.034. This means that any autocorrelation of more than 0.07 is highly statistically significant. Similarly, if you’re using daily data points for that period, then any autocorrelation above 0.015 is significant. On the other hand, a contrarian’s juices are stimulated by poor returns over several years. As noted above, you can divide the 72 years from 1926 to 1997 into 18 periods of four years each. This means that you’ll need an autocorrelation of -0.44 to establish statistical significance. (Negative autocorrelations define contrarian strategies: a good return in one period forecasts a higher probability of a poor one in the next.) Put another way, an autocorrelation of -.08 (similar to that seen with monthly periods) would require 3500 years of data to attain the same degree of statistical significance.
What this all means is that contrarian strategies are essentially untestable, and if we want to disprove the random-walk hypothesis, we are stuck with testing for momentum.

The Historical Data
A nice summation of the autocorrelation data for US stocks is found in Campbell, Lo, and MacKinlay’s ("CLM") The Econometrics of Financial Markets. The following table summarizes their autocorrelation data for 1962 through 1994:

CRSP Value Weighted
("large stocks")
CRSP Equally Weighted
("small stocks")
Daily Returns
Weekly Returns
Monthly Returns

CRSP refers to the Center for Research in Security Prices. The value-weighted and equally-weighted indexes can be very roughly thought of as large and small stock proxies, respectively.

This data pretty conclusively demonstrates momentum effects of high statistical significance for an index of large stocks from day to day, but not for longer periods. An index of small stocks does demonstrate momentum over days, weeks, and months. (I wouldn’t get too excited over the 0.350 autocorrelation for small stocks for daily periods. Remember that many of these securities do not trade every day, so that a big market move up or down one day will be followed by appropriate moves in ensuing days in the stocks that did not trade.)

In light of the above, it is rather amazing that when CLM looked for momentum in individual stocks, none was found. In other words, the generations of investors who have been gazing at stock price charts likely have been wasting their time, but the recent phenomenon of charting mutual fund prices may have some validity. CLM explain this apparent paradox by noting that there are highly significant "cross autocorrelations" between large and small stocks, meaning a rise/fall in large stocks is usually followed by a rise/fall in small stocks.

Foreign Markets

What about non-US bourses? It’s a good news/bad news story. The good news is that there are dozens of them out there to look at. The bad news is that their historical record is considerably shorter, some less than 11 years. Oh, and one other problem. The data is very hard to get, unless your name is Morgan Stanley. Still, I was able to scrounge a fair amount of relevant data from Morningstar’s Principia Plus®. The US, UK, and Japanese data were obtained from Dimensional Fund Advisors. Here’s the data for autocorrelations of monthly returns:

Country Number of Months Autocorrelation
p value
Japan (large)
Japan (small)
U.K. (large)
U.K. (small)
U.S. (large)
U.S. (small)

All in all, this table provides pretty impressive evidence of momentum abroad. Consider that of the 26 non-US markets studied, all but six had positive autocorrelations, six had p values which reached the .05 level of significance, and another six which reached it at a 0.1 level, whereas by chance we would have expected only one of each. Second, note that the longer the historical record, the more impressive the statistical power, particularly the US and UK. In fact, it is rather amazing that Philippines reached the .05 level of significance with only 129 data points (10.75 years).

What’s It All Mean?

OK, so stocks around the world do not do the random walk. How does this data affect the average investor? Only at the margins. Lest we get too carried away, the most impressive autocorrelations we’ve encountered are in the 0.2 range. That means that no more than 4% (0.2 squared, or "R-squared") of tomorrow’s price change can be explained by today’s. That doesn’t buy a lot of yachts. For the taxable investor, this stuff is totally irrelevant—whatever advantage there is to this technique is obliterated by the capital gains capture mandated by buying and selling with the high frequency necessitated by momentum techniques.

Certainly, however, these effects cannot be ignored. For the sheltered asset allocator, the message is loud and clear: Do not rebalance too frequently. If asset class prices have a tendency to trend over relatively long periods (say months, or even one to two years) then rebalancing over relatively short periods will not be favorable. This is a somewhat tricky concept. Remember that asset variance (which is the square of the standard deviation) is one of the main engines of rebalancing benefit. If an asset has momentum, then the annualized variances will be greater over long periods than over short periods—this is in fact a good way to test for momentum.

Think about the Japanese and US markets. Both have exhibited pretty impressive momentum (in opposite directions) since 1989. Obviously, rebalancing as little as possible from the US to Japan would have been more advantageous than doing it frequently.

Yet another way of thinking about this is the following paradigm—rebalance only over time periods where the average autocorrelation of your assets is zero or less. For practical purposes, this means no more than annually, and preferably less.

Yin, Yang
Rather than being polar opposites, momentum investing and fixed asset allocation with contrarian rebalancing are simply two sides of the same coin. Momentum in foreign and domestic equity asset classes exists, resulting in periodic asset overvaluation and undervaluation. Eventually long-term mean reversion occurs to correct these excesses.

Over two decades ago, Eugene Fama made a powerful case that security price changes could not be predicted, and Burton Malkiel introduced the words "random walk" into the popular investing lexicon. Unfortunately, in a truly random-walk world, there is no advantage to portfolio rebalancing. If you rebalance, you profit only when the frogs in your portfolio turn into princes, and vice versa.

In the real world, fortunately, there are subtle departures in random-walk behavior which both the asset allocator and momentum investor can exploit. Writer/money manager Ken Fisher calls this change in asset desirability, and the resultant short term-momentum and long-term mean reversion, the "Wall Street Waltz."

Even investors who eschew momentum techniques should be aware that momentum exists. Understanding what it means for rebalancing and asset behavior will make you a better asset allocator.

To Efficient Frontier 
Homepage E-mail To William Bernstein

copyright (c) 1999, William J. Bernstein