Data quality: survivorship bias, adjustments and gaps

What you will learn

• What survivorship bias is and how much it can inflate a backtest.
• How splits and dividends are adjusted, and when to use price or total return data.
• How continuous futures series are built and why they can show odd prices.
• How to find and handle gaps, bad prices and other data errors.

Explanation

A backtest is only as good as its data. Most data errors make results look better, not worse, so they are dangerous.

Survivorship bias

Survivorship bias means testing only on instruments that still exist today. Companies that went bankrupt, were taken over or were removed from an index disappear from many free data sets. A stock-picking system tested on today's index members has, in effect, been told in advance which companies would survive. The fix is point-in-time data: the universe as it was known on each historical date, including delisted securities (removed from the exchange) and their final prices.

Corporate actions and adjustments

• Stock split: one share becomes several, and the price falls by the same factor. Unadjusted data show a fake crash. Adjusted data divide all earlier prices by the split ratio.
• Dividend: on the ex-date (the first day the buyer no longer receives the dividend) the price usually drops by roughly the dividend. A price series shows this as a loss; a total return series adds the dividend back. Adjustment factor for earlier prices: 1 − dividend ÷ previous close.

Use total return (adjusted) data to measure returns, but remember that adjusted historical prices are not the prices that actually traded. A rule such as "buy above €100" must use the real (unadjusted) price of that day.

Futures: continuous series

Futures contracts expire, so a long history must join several contracts (rolling was covered in the course "Derivatives: Futures, Options, CFDs and Certificates"). At each roll the old and new contracts have different prices. A back-adjusted series shifts all earlier prices by the roll gap so the joined series has no artificial jump. The difference method adds or subtracts the gap in euros or dollars and can produce negative historical prices; the ratio method multiplies by a factor and keeps percentage moves correct. Calculate returns in percent from ratio-adjusted series, or from each real contract.

Gaps and errors

• Missing bars: holidays, trading halts or a data hole. Do not silently fill them with the previous price; mark them and decide a rule (lesson 2).
• Bad prices: a single wrong value, for example a price 10 times too high. Check for one-day moves above a threshold (such as 25% for a broad index) and confirm them against a second source.
• Timing mismatches: markets close at different times. A European ETF's close and a US index close are not simultaneous.
• Revised data: economic figures and company accounts are later corrected. Use the value that was published at the time, not the revised one.

Worked example

Hypothetical example 1, split: a share closes at €400. Overnight it splits 4-for-1 and next day closes at €101.

• Unadjusted return: €101 ÷ €400 − 1 = −74.75% (fake crash).
• Adjusted previous close: €400 ÷ 4 = €100. Return: €101 ÷ €100 − 1 = +1.00%.

Hypothetical example 2, dividend: close before the ex-date €50.00, dividend €1.00, ex-date close €49.50.

• Price return: €49.50 ÷ €50.00 − 1 = −1.00%.
• Total return: (€49.50 + €1.00) ÷ €50.00 − 1 = +1.00%.
• Adjustment factor: 1 − €1.00 ÷ €50.00 = 0.98. Adjusted previous close: €50.00 × 0.98 = €49.00. Return from adjusted data: €49.50 ÷ €49.00 − 1 ≈ +1.02%. The small difference comes from the method, which assumes the dividend is reinvested at the ex-date price.

Hypothetical example 3, survivorship: a universe of 10 stocks over five years. Eight still exist, each up +40%. One went bankrupt (−100%), one was delisted after −80%.

• Survivors only: average +40%.
• Full universe: (8 × 40% − 100% − 80%) ÷ 10 = (320% − 180%) ÷ 10 = +14%.
• The survivor-only test overstates the average by 26 percentage points. Fees and taxes are excluded.

Common mistake

Practice activity

Hypothetical data: a stock closes at €90. It splits 3-for-1 and next day closes at €31. Two weeks later it closes at €32.00 before an ex-date; the dividend is €0.64 and the ex-date close is €31.80. Calculate (a) the adjusted return on the split day, (b) the price return and total return on the ex-date, and (c) the dividend adjustment factor.

Key takeaways

• Survivorship bias inflates results; use point-in-time universes including delisted securities.
• Measure returns with split- and dividend-adjusted data, but apply price-level rules to real prices.
• Ratio back-adjustment keeps percentage moves of futures series correct; difference adjustment can create negative prices.
• Check for missing bars, extreme one-day moves and revised data before running any test.

Next step

In the next lesson you will build a complete backtest in a spreadsheet, column by column, using clean adjusted data.