Backtesting: what it can really tell you (and how it can fool you)

There is a moment, right before real money moves, when every portfolio idea looks brilliant. Backtesting exists for that moment: it takes your rules — this allocation, this rebalancing cadence, this method — and replays them through the past, day by day, to see what would have happened. It is the closest thing to a laboratory that personal finance has to offer.

What it actually produces

A backtest returns an equity curve (how the portfolio’s value would have evolved), the drawdowns (declines from the previous peak: how deep, how long) and the summary metrics — annualized return, volatility, return-to-risk ratio. Read together, they tell you not just how much a strategy would have made, but what you would have had to sit through to get it: a −30% that lasts two years is psychological information before it is statistical.

The real value: falsifying ideas

A backtest is not there to confirm you are right. It is there to try to prove you wrong at zero cost: an idea that collapses over ten years of data has just spared you the experiment with your own savings. It is the scientific method applied to a portfolio — and as in science, a passed test does not prove the theory. It merely leaves it not yet refuted.

The four traps

This is where the backtest turns from instrument into trap, because the past is a corruptible examiner.

• Overfitting — if you keep tuning parameters until the curve looks perfect, you have not discovered a rule: you have memorized the noise of that particular decade. The more a strategy is optimized for the past, the more fragile it tends to be in the future.
• Survivorship bias — testing only securities that still exist today means interviewing only the survivors: the failures have left the sample, taking the uncomfortable half of history with them.
• Look-ahead bias — using, even accidentally, information that was not available at the time (an earnings report published later, a closing price used to trade that same day). Future data leaking into the past inflates any result.
• Ignored costs and frictions — commissions, spreads, slippage and taxes turn many paper “winners” into real-world break-evens. A backtest without costs is an optimistic hypothesis, not a simulation.

How to read one like an adult

Look for robustness, not perfection: if nudging a threshold or shifting the start date changes the outcome dramatically, you were not looking at a strategy — you were looking at a coincidence. Distrust curves that are too smooth. Weigh the drawdowns more than the final return. And remember the sample size is one: the past that happened, out of the many that could have.

What it is NOT

A backtest is not a forecast, and its return is not a promise: it is the behaviour of one rule in one specific past, under simplifying assumptions. It does not tell you what to buy or when — it describes, it does not prescribe. The decision, and the risk, remain yours.

A backtest can’t tell you that you’re right. It can tell you that your idea has already been proven wrong — which is more than most opinions ever learn.