What returns explain and what they do not: Insights from 150 years of market data

Stock prices and company profits move closely together over long periods of time, a relationship confirmed by greater than a century of information compiled by Robert Shiller. This evaluation examines the strength of this long-term connection and tests whether changes within the earnings-price correlation provide insight into future stock market returns.

The results show that while profits help explain market behavior over time, fluctuations in correlation themselves don’t provide a useful basis for predicting returns. The following sections document empirical patterns across multiple rolling time periods and evaluate the restrictions of using correlation measures as market timing tools. The results may also help financial advisors determine long-term market behavior for clients in an informed and intuitive way.

What this evaluation is meant to make clear

I examine the long-term relationship between stock prices and company earnings for 2 fundamental reasons.

First, the outcomes provide a straightforward option to explain stock market behavior over long investment horizons. I define a protracted horizon as greater than 10 years, which is a useful minimum period for retirement planning and asset allocation decisions.

Second, after calculating the correlations between prices and returns, I tested whether changes within the correlation over time could function a number one indicator of future returns. Specifically, I asked whether periods of unusually low historical correlation were followed by stronger or weaker subsequent stock market performance.

Correlation results

The evaluation uses monthly averages of S&P Composite earnings per share and S&P Composite price. Reported monthly earnings, share price and return data for the S&P Composite corporations is predicated on Shiller’s data from 1871 to December 2024.

Over several periods, correlations between earnings and costs have been consistently high.

I selected common time periods to look at the info and located the next:

• A start line is the Investors Act of 1940, which goals to check whether outcomes differed after the introduction of investor protections and more uniform accounting standards. The difference seems negligible.
• The last 10 and 20 years were included to reflect what is usually considered a typical retirement planning horizon.

Correlation changes over time

The correlation between earnings and stock prices fluctuates over time, particularly over shorter time horizons similar to the five-, 10- and 20-year periods. The rolling 50-year correlations also vary, although over a much narrower range.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

The lowest 50-year rolling correlation occurred in the primary half of the twentieth century when the info series reached 0.6. Against the background of two world wars, the Great Depression and limited market regulation before 1940, it’s noteworthy that the correlation didn’t decrease further.

As the time horizon shortened, the variability increased. In the rolling 20-year series, correlations fell below 0.50 for a full decade between February 1918 and December 1928, and again briefly fell below 0.50 in December 1948.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

The 10-year rolling correlations fell below zero in three periods: at the top of World War I and World War II and throughout the hyperinflation era of the late Seventies and early Eighties.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

Five-year rolling correlations naturally showed essentially the most volatility, with deeper declines and more frequent swings, including multiple periods of negative correlation. Both the typical and median five-year rolling correlations were lower than correlations observed over longer periods.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

Does the variability of the correlations correspond to the returns?

To test whether variations in earnings-price correlation have any predictive value for stock returns, we ran regressions of correlation levels versus trailing annualized returns.

The R² between the S&P Composite’s earnings and the value from 1871 to 2024 could be very high at 0.95. Given the strength of this long-term relationship – and the relative rarity of periods of low correlation – it is affordable to ask whether these periods could act as buy or sell signals. In other words, do variations in earnings-price correlation help predict future returns?

I evaluated this query over several rolling time horizons. The resulting R² values ​​– which link correlation levels to subsequent annual returns – were far lower than the R² between earnings and price itself. For the rolling 10-year and five-year windows, the R² fell near zero, suggesting that there’s virtually no predictive relationship.

The rolling 50-year period showed the strongest relationship with an R² of 0.53.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

For the 20-year rolling windows, the R² was 0.24, reflecting significantly greater variability.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

The variability continued to extend within the 10-year rolling series, where the R² fell to 0.06.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

The rolling five-year periods don’t show a consistent pattern. R² is nearly 0.0 (actually: 1.27E-07).

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

Overall, I discovered no evidence that changes in earnings-price correlation predict future annual returns. The data show that the 2 measures don’t fit together meaningfully for time horizons of lower than 50 years.

Predictive power of correlation

The strong long-term relationship between earnings and costs provides a transparent explanation for the rise and fall of stock markets over longer periods of time. It provides a straightforward and intuitive framework for understanding long-term stock trends.

However, the second goal – determining whether changes in correlation could function a predictor of annual returns – was not achieved. The evidence suggests that aspects apart from the earnings-price relationship determine the speed of change in annualized returns, even when the 2 series are close over long periods of time.

Key insights

• Earnings and stock prices move closely together over long periods of time. More than 150 years of Shiller data show a consistently strong relationship between the 2 series.
• Shorter windows cause significant noise. Correlations vary significantly over five, ten and 20 12 months periods, reflecting wars, inflation shocks and structural changes.
• Correlation strength doesn’t imply predictive power. Shifts in earnings-price correlation are unlikely to predict subsequent returns over the time horizons relevant to most investors.
• Only the longest windows have limited informative value. Even the 50-year regressions provide only modest insights with an R² of 0.53, while shorter time horizons are near zero.

Profits help explain long-term market behavior, but they don’t help time the market.

.