From backtest to live: monitoring and when to stop a system
Move a tested system into live trading in stages, measure implementation shortfall, and use pre-defined statistical and drawdown rules to decide when to pause or stop.
What you will learn
- A staged path from backtest to full live trading
- How to measure implementation shortfall between backtest and live fills
- How to judge live results with the standard error of expectancy
- How to write pause and stop rules before the first live trade
Explanation
Expectancy, R-multiples and losing-streak mathematics are covered in the risk management course. This lesson uses them to monitor a running system.
A staged start
- Incubation (paper trading): run the system in real time without money for a fixed period, for example three months. You check that signals, data and order instructions work exactly as in the backtest.
- Small live allocation: trade with a reduced size, for example a quarter of the planned risk. Real fills, fees and rejected orders now appear.
- Full allocation: increase size only when the pre-defined checks are passed.
Completing these stages does not prove the system has an edge; it only shows that the implementation matches the test.
Implementation shortfall
Implementation shortfall is the gap between the backtest result and the live result for the same signals. Compare each live trade with the price the backtest would have used. A growing shortfall usually comes from slippage, wider spreads, delays or data differences. If it is larger than the costs assumed in your test (lesson 8), update the cost model and re-check whether the system is still positive.
Is the live result really worse?
Short live periods are noisy. The standard error (SE) tells you how uncertain an average is:
SE = standard deviation of trade results ÷ √(number of trades)
Then calculate how many standard errors the live average is away from the backtest expectancy:
z = (live average − backtest expectancy) ÷ SE
As a rough guide, a z-value between −2 and +2 means the live result is still consistent with the backtest. A value below −2 is a warning that the edge may be smaller or gone. This assumes independent trades with roughly stable behaviour, which real markets do not guarantee.
Pause and stop rules
Write these rules before going live, so that emotions after a loss do not decide:
- Drawdown limit: pause if the live drawdown (fall from the highest equity) exceeds, for example, 1.5 × the maximum drawdown seen in the backtest. Historical drawdowns are not a limit on future ones.
- Statistical check: after at least 30 live trades, review if z is below −2.
- Operational checks: missing data, wrong prices after corporate actions such as splits, unfilled orders, or broker changes to fees and margin. Stop immediately if the system is not running as specified.
- Market change: if the market changes in a way the system was never tested for, for example a new trading-hours regime or a product change, pause and re-test.
After a pause, the choices are: resume unchanged, resume at smaller size, or retire the system. Do not re-optimise on the recent losing period alone; that is overfitting to the latest data.
Worked example
Hypothetical example: a backtest shows an expectancy of +0.25R per trade with a standard deviation of 1.2R and a maximum drawdown of 12%. After 40 live trades the average result is +0.02R and the account is 9% below its peak. Fees are included in R; taxes are excluded.
Step 1: SE = 1.2 ÷ √40 = 1.2 ÷ 6.32 ≈ 0.19R.
Step 2: z = (0.02 − 0.25) ÷ 0.19 ≈ −1.2.
Step 3: z is above −2, so the live result is disappointing but still within normal variation.
Step 4: drawdown limit = 1.5 × 12% = 18%. The current 9% is below the limit.
Step 5: decision by the written rules: continue unchanged and review again later.
How many trades would give an SE of 0.1R? n = (1.2 ÷ 0.1)² = 144 trades. A clear answer needs far more trades than most people expect.
Implementation check: the backtest average trade for the same 40 signals was +0.60% and the live average +0.45%. Shortfall = 0.15 percentage points, which is 0.15 ÷ 0.60 = 25% of the backtest edge. This deserves attention even though the statistical test is not yet alarming.
Common mistake
Practice activity
On paper: a backtest shows +0.30R expectancy, a standard deviation of 1.5R and a maximum drawdown of 16%. The written rule says: pause if drawdown exceeds 1.5 × backtest maximum, or if z is below −2 after at least 30 trades. After 25 live trades the average is −0.10R, and equity fell from a peak of €26,000 to €21,500. (a) Calculate SE and z. (b) Calculate the current drawdown and the limit. (c) What do the rules say?
Show solution
(a) SE = 1.5 ÷ √25 = 0.30R. z = (−0.10 − 0.30) ÷ 0.30 ≈ −1.33.
(b) Drawdown = (€26,000 − €21,500) ÷ €26,000 ≈ 17.3%. Limit = 1.5 × 16% = 24%.
(c) Neither rule is triggered (fewer than 30 trades, z above −2, drawdown below 24%). Continue, and keep checking implementation shortfall.
Limitations: the z-test assumes independent, stable trade results; real results can cluster.
Key takeaways
- Go live in stages: paper trading, small size, then full size.
- Measure implementation shortfall trade by trade against the backtest.
- Use the standard error to separate normal variation from a real decline.
- Write drawdown, statistical and operational stop rules before the first live trade.
Next step
In the final lesson you will put every step of this course together in one complete systematic-trading project on paper.
Self-check
Answer the questions, then check your answers. You can retry as often as you like.
Automatic self-check for your own revision. Your answers are checked in your browser and are not seen or assessed by us. It is not an exam and not a qualification.
Your progress
“Complete” only means you marked the lesson as done. It is not a test of your understanding. Progress and notes are saved only in this browser on this device — not in an account.