Advertised Backtest Check
Free backtest overfitting calculator: enter an advertised Sharpe ratio, the length of the record and how many variants were tried, and see whether the claim beats luck.
Before you buy a strategy, an EA or a signal service, run its advertised numbers through this backtest overfitting calculator. It shows the Sharpe ratio that pure chance would have produced after that many attempts — and how many live years the claim would need to prove itself.
QuantDojo · Free tool
Advertised Backtest Check
Someone is selling you a strategy, an EA or a signal service. Type the numbers off their sales page and find out whether the claim survives the one question the page never answers: how many variants were tried before this one looked good?
How it works: the advertised Sharpe is compared against the best Sharpe you would expect from luck alone after that many tries (Bailey & López de Prado's Deflated Sharpe Ratio), then corrected for skew and fat tails. “Live years needed” is the minimum track record length to separate the claim from that luck threshold at 95% confidence. No Sharpe on the sales page? Compute it first with the Sharpe ratio calculator.
Educational tool — not financial advice. The vendor almost never states how many variants were tested, so that field is your assumption: try 1 and 1,000 and see how little the advertised number survives. Assumes the tested variants differ only by chance (their Sharpes scatter with the sampling error of the record length).
Is your backtest lying to you?
Get The Honest Backtest Checklist — the exact red flags that expose an overfit strategy before you risk a cent, plus one rigorous idea a week.
Free forever. Unsubscribe anytime. We test what others sell — no hype. · Prefer to read it now? →
How to use the advertised backtest check
Take the numbers straight off the sales page: the Sharpe ratio, how many years the record covers, and how often it is measured. Then set the one field the seller never fills in — how many variants were tried before this one made the screenshot. Ten thousand for anything that calls itself AI-selected or comes out of an optimiser; a hundred for a hand-tuned system; one only if the seller published the rules before the results existed.
The calculator returns three things: the Sharpe ratio you would expect from luck alone after that many attempts, the probability the advertised edge is more than that, and how many live years the record would need before the claim could be told apart from chance.
The instructive move is to change one field. Leave the advertised numbers alone and raise the variant count from 1 to 1,000. Most sales pages fall apart between those two settings — which is the whole point: the headline number was never a property of the strategy, it was a property of the search.
Frequently asked questions
Why does the number of tested variants matter so much?
Because the Sharpe ratio has no memory of the search. Test one strategy and a Sharpe of 2 is remarkable. Test ten thousand on the same data and a Sharpe of 2 is the expected best result even when not one of them has any edge at all. The advertised figure is always the winner of a competition you were not shown.
What is a deflated Sharpe ratio?
The Deflated Sharpe Ratio (Bailey and López de Prado) compares an observed Sharpe against the best value you would expect from the number of trials that produced it, corrected for the length of the record and for skew and fat tails. It answers "is this better than the luckiest of my attempts?" instead of "is this better than zero?".
The sales page shows a 13-year chart. Isn't that long enough?
Only if those 13 years were live — published in advance, with no chance to revise them. A backtest is the place where the tuning happened, so its length measures how much data the search had to fit, not how much evidence you have. In our ProPicks vendor check the backtest era and the live era of the same strategy pointed in opposite directions.
The page doesn't state a Sharpe ratio. What now?
If it gives an average return and a volatility, compute the Sharpe with the Sharpe ratio calculator first. If it gives neither — only a rising curve and a total return — that absence is itself the finding: a seller who will not publish a risk-adjusted number has decided you should not compute one.
Does a high probability mean the strategy will make money?
No. It means the record is hard to explain by chance under the assumptions you entered. Costs, slippage, capacity, regime change and the seller's honesty about the trial count are all outside this calculation.
Got the actual return series rather than just the sales page? Run it through quantcheck for the full multiple-testing check — deflated Sharpe, probability of backtest overfitting and a bootstrap, free and without an account.