Survivorship Bias in Backtesting

Survivorship bias occurs when a backtest only includes assets that exist today, ignoring those that failed, delisted, merged, or went bankrupt during the test period. You’re testing on the “survivors” and ignoring the dead.

If you backtest a stock-picking strategy on the current S&P 500 constituents back to 2005, you’re only testing on companies that survived and thrived enough to still be in the index. Lehman Brothers, Enron, WorldCom, Bear Stearns — all invisible to your backtest. In real-time, your strategy would have had to deal with them.

It’s like surveying only successful restaurateurs and concluding that restaurants are a great business. You never talked to the 80% that went under.

How Survivorship Bias Inflates Performance

The bias inflates backtest results in multiple ways:

  • Upward bias in returns: Stocks that survived performed better on average than those that didn’t. Your universe is pre-filtered for winners.
  • Understated drawdowns: Stocks that went to zero during crashes (Bear Stearns, Lehman) are absent. Your portfolio would have held some of these — those losses don’t appear in the backtest.
  • False diversification: Testing on survivors makes it look like diversification always works. During crises, failed companies create concentrated losses that your backtest never sees.
  • Magnitude: Studies estimate survivorship bias inflates annual stock returns by 0.5% to 1.5% per year. Over 20 years, that compounds to 10-35% of overestimated cumulative return.

Concrete Examples

S&P 500 Backtest

You backtest a momentum strategy on the current S&P 500 constituents from 2000-2023. Enron (2001), WorldCom (2002), Lehman Brothers (2008), and dozens of other companies that were in the S&P 500 at the time are missing. In real-time, some of these would have been in your portfolio before they collapsed.

Your backtest never buys a stock that goes to zero because those stocks aren’t in the dataset.

Mutual Fund Selection

You test a strategy picking top-performing mutual funds from the past 3 years and rotating into them. Using a current fund database, results look excellent. But poorly performing funds get merged or closed by their parent companies, removing bad track records. All surviving funds look better than they actually were.

Crypto Backtesting

You backtest a momentum strategy across “all cryptocurrencies” using today’s list. But thousands of tokens from 2017-2018 that went to zero aren’t in your data. Your backtest never buys a coin that goes to zero because those coins no longer exist in the database.

This is particularly severe in crypto where failure rates are extremely high.

Country-Level Bias

Backtesting a global equity strategy using only countries with surviving, functional stock markets (US, UK, Japan). Ignoring markets that were interrupted or destroyed (Russia 1917, China 1949, many emerging markets). US stocks have had exceptional returns partly because the US was exceptionally successful — testing only on the US introduces survivorship bias at the country level.

Impact by Strategy Type

Strategy TypeSurvivorship Bias Impact
Large-cap momentumModerate (1-2% annual)
Small-cap valueHigh (2-4% annual)
S&P 500 buy-and-holdLow (0.5-1% annual)
Distressed/turnaroundVery high (5%+ annual)
Pairs tradingModerate to high
Crypto strategiesVery high

Value strategies are hit hardest because they buy cheap, distressed stocks — many of which go bankrupt. If bankruptcies are removed from the data, value looks much better than it really is.

How to Prevent Survivorship Bias

Use Bias-Free Databases

This is the most important step. Sources that include delisted securities:

  • CRSP (Center for Research in Security Prices) — the gold standard for US equities, used by virtually all academic research
  • Compustat Point-in-Time — fundamentals data without forward-looking bias
  • Norgate Data — affordable survivorship-bias-free data for retail traders
  • Sharadar/Quandl — includes delisted stocks
  • Tiingo — includes some delisted equity data

Use Point-in-Time Index Constituents

Don’t backtest on today’s S&P 500 members. Use historical constituent lists showing which stocks were actually in the index at each date. Many data providers offer this.

Include Delisting Returns

When a stock delists, its final return matters. CRSP assigns a delisting return (often -30% to -100% for involuntary delistings). Without this, you assume delisted stocks vanish at their last traded price — which is wrong.

Test on Broad Universes

Testing on all US stocks above $5 with survivorship-bias-free data is better than testing on a curated index.

Know Your Data

Ask explicitly: “Does this data include delisted securities?” If the answer is no or unclear, assume survivorship bias is present.

How to Detect Survivorship Bias

  • Compare against known benchmarks: If your buy-and-hold backtest on “the S&P 500” outperforms the actual S&P 500 total return index, you likely have survivorship bias.
  • Check for low bankruptcy rates: In real markets, stocks get delisted regularly. If your universe never shrinks, something is wrong.
  • Look for missing names: If your 2008 data doesn’t include Bear Stearns, Lehman Brothers, Washington Mutual, or Wachovia, you have survivorship bias.

Resources