A Vectorised Backtest: Signals, Positions, Shifting to Avoid Look-Ahead, Transaction Costs and the Equity Curve

What you will learn

• The five-column structure of a vectorised backtest: signal, position, asset return, cost and strategy return.
• Why shift(1) on the position column is the single line that separates an honest test from a look-ahead fantasy.
• How to charge proportional transaction costs on position changes and build an equity curve in euros.
• What a vectorised test can and cannot model, and when the event-driven engine of lesson 8 is needed.

Explanation

A vectorised backtest computes the whole history at once with column operations instead of looping bar by bar. The spreadsheet backtest from the Systematic Trading course had this structure; pandas runs it over decades in milliseconds. It suits long-flat or always-in rules acting on the close; intrabar stops and specific fill prices are lesson 8's job.

Step 1, signal. signal = (close > sma(close, 50)).astype(int) gives 1 when the close is above its 50-day average, else 0. It describes what you know at the close of bar t.

Step 2, position. You cannot trade on a close you have not yet seen; the earliest you can hold bar t's signal is bar t + 1, so position = signal.shift(1).fillna(0). Row t of the position column now means "the exposure held during bar t", decided at the end of bar t − 1. Omitting the shift lets the strategy earn bar t's return using bar t's close, which is look-ahead bias: using information unavailable at decision time.

Step 3, asset return. ret = close.pct_change().fillna(0) (lesson 5). Because the position during bar t was fixed at bar t − 1's close, position * ret is the gross strategy return for bar t. The implicit assumption is a fill at the close of bar t − 1; lesson 8 handles fills at the next open.

Step 4, costs. Commission, spread and slippage are modelled as a proportional rate per unit of position changed: turnover = position.diff().abs().fillna(0) is 1 when going flat to long or back, 2 on a reversal. cost = turnover * c, with c the one-way cost as a fraction (0.001 for 0.1%). The cost enters the return column on the bar where the position changes.

Step 5, strategy return and equity. strat = position * ret - cost, and equity = capital * (1 + strat).cumprod(). The buy-and-hold benchmark is capital * (1 + ret).cumprod(). Always compare both with the same metrics (lesson 9); merely diluting buy-and-hold with cash lowers drawdown and return alike, which is not skill.

State the assumptions: exposure is a fraction of equity reinvested daily, fills at the close, proportional costs, no interest on cash, dividends only via adjusted data, taxes ignored. The test then answers one narrow question: did the timing rule add anything over holding the asset, after costs?

Checks before trusting the output. (1) Set cost to zero and the signal to a constant 1; equity must equal buy-and-hold exactly. (2) Count round trips with turnover.sum() / 2 and check plausibility. (3) Make sure the indicator's warm-up NaN rows give a zero position, not a spurious trade.

Worked example

Hypothetical example: eight closes in euros, 100, 101, 103, 102, 104, 103, 101, 100; long when the close is strictly above its 3-day SMA, else flat; one-way cost 0.1%; capital €10,000; no interest, dividends or taxes.

Bar 3: the signal at bar 2 was 1 (103 > 101.33), so the position during bar 3 is 1. The asset fell 103 → 102 (−0.971%), turnover 1 costs 0.1%, strategy return −1.071%, equity 10,000 × (1 − 0.01071) = €9,892.91. At bar 3 the close equals the SMA, so the strict rule gives 0 and the position is closed during bar 4 (cost 0.1%, no exposure). Final equity €9,768.33 (−2.32%) versus buy-and-hold of exactly €10,000. Costs took €39.36 over four one-way trades.

Now the look-ahead version, using signal instead of the shifted position: the strategy "holds" during bars 2 and 4, the two up days it could only identify after they closed, and ends at €10,356.65 (+3.57%). One missing shift(1) turned −2.32% into +3.57%.

Common mistake

Practice activity

On paper, with the same eight closes and rule: (a) recompute the final equity with a one-way cost of 0.2%; (b) state the number of round trips; (c) a colleague proposes filling the SMA's first NaN values with the close so that "trading can start on day one"; explain briefly why this is a problem.

Key takeaways

• A vectorised backtest is five columns: signal, position (shifted), asset return, cost on turnover, strategy return; equity is the cumulative product.
• position = signal.shift(1) is non-negotiable; test by removing it and watching the result inflate.
• Charge costs on position.diff().abs() and always compare against buy-and-hold with identical assumptions.
• Vectorised tests suit close-to-close rules; stops, sizing and realistic fills need the event-driven engine of lesson 8.

Next step

Lesson 8 builds an event-driven backtest: a bar-by-bar loop with ATR stops, position sizing and a trade list measured in R-multiples.