The Hidden Force Behind Market Cycles: Investor Disagreement
By James Picerno | The Milwaukee Company
New research provides strong evidence that investor beliefs—not fundamentals—drive long‑term market dynamics
Sophisticated investors tend to be contrarian, while retail investors often extrapolate past returns.
Valuation ratios like CAPE reflect sophisticated—not retail—expectations about future returns
How do financial markets price stocks? The answer matters because it provides information for designing and managing investment strategies.
A dominant line of analysis in asset pricing argues that long-term stock market fluctuations are driven by fundamentals like time-varying risk premia. However, a new research paper challenges this understanding by modeling the market as an interaction between naive investors (whose beliefs are decoupled from fundamentals) and sophisticated investors, using a new 70-year dataset to support the findings.
A key takeaway is that shifts in valuation ratios are almost entirely driven by expected changes in investor sentiment and price mean-reversion, not by changing expectations of dividend or earnings growth.
The paper also provides more context and support for engaging in tactical tilts at times. The standard academic and industry view—rooted in the Efficient Market Hypothesis (EMH)—holds that markets are broadly “efficient,” making buy‑and‑hold, market‑cap‑weighted indexing the logical baseline. Volatility, in this framework, simply reflects “rational” investors updating expectations. The EMH has been hard to beat over long horizons, but behavioral finance has steadily shown that real‑world investing is more complicated. Markets may be efficient enough to justify indexing, yet not so efficient that tactical adjustments are entirely pointless.
A new research paper strengthens this case for periodically adjusting portfolios beyond what a simple buy-and-hold strategy implies. In Beliefs and Stock Market Fluctuations: New Evidence from the Past Seven Decades, MIT Sloan economists David Thesmar and Emil Verner analyze seven decades of data and uncover three central findings about how different investors form expectations.
Sophisticated Investors Are Contrarian; Retail Investors Extrapolate
The idea of a unified “market consensus” is misleading. Beliefs are fragmented—and often move in opposite directions. Retail investors, as shown in surveys, are highly procyclical and extrapolative: when markets rise, they assume gains will continue and their expected returns peak near market tops. Value Line analysts, who are used in the paper as a proxy for “sophisticated” investors, behave very differently. Their expectations are countercyclical: when markets fall, they anticipate higher future returns.
Valuation Ratios Reflect Sophisticated, Not Retail, Expectations
Valuation metrics, such as the cyclically adjusted earnings-price ratio (CAPE) and earnings-price (E/P) ratio, have long been known to predict long‑term returns. Thesmar and Verner show whose beliefs they actually capture. A time‑series regression of the market’s earnings-price (E/P) ratio on Value Line’s expected returns produces an R² of 65%. Low valuations tend to align with high expected returns among sophisticated investors—exactly what rational models predict. Retail surveys generally show no such relationship. A Campbell‑Shiller decomposition reinforces the point, the study notes:
Roughly 60% of variation in price-earnings ratios stems from shifts in sophisticated investors’ expected returns.
0% comes from expected changes in cash flows or dividend growth.
A graphic in the study captures the point, showing that the Value Line expected returns “account for a large share of movements in valuation.”
Disagreement Drives Volume and Volatility
If sophisticated investors are closer to rational, why do bubbles and crashes still occur? Limits to arbitrage. When retail optimism surges, naive investors buy aggressively, pushing prices above fundamentals. Sophisticated investors recognize the mispricing but cannot fully counteract it due to risk constraints. The resulting expectation gap fuels trading. When disagreement between Value Line and retail investors spikes, market volume and volatility jump as the two groups trade against each other.
The authors report that retail investors at times persist in pushing prices away from fundamentals, while sophisticated investors trade against them and can forecast future returns. The historical record, in other words, supports contrarian‑leaning strategies at times. In the jargon of financial researchers, “heterogeneous beliefs matter for market dynamics,” the paper notes.
Theory and Practice
This is old news for some investors. The likes of Warren Buffett and other value investors have been profitably exploiting this behavioral gap for decades. By some accounts the Oracle of Omaha’s impressive track record is an anomaly. Thesmar and Verner’s research, however, suggests there’s a recurring behavioral risk factor embedded in the stock market that can be used for tactical tilts, even within an indexing‑first philosophy.
Markets are smart, but they aren’t flawless. While low-cost indexing remains the smartest anchor for most portfolios, this research reminds us that human nature never changes. For the indexing-first investor, that means the occasional tactical tilt may not just be a gamble—recent research increasingly suggests it’s a data‑driven response to a persistent behavioral reality.
