What a PocketX backtest actually proves — and what it does not

A backtest is the most persuasive and most misread artefact in rule-based trading. It produces a number, the number looks like a result, and the mind quietly converts "this is what the rule did" into "this is what the rule will do".

PocketX shows backtest evidence rather than a verdict, and it states the assumptions that produced it. This article explains how to read that evidence without deceiving yourself.

A backtest is a conditional statement

Every backtest is really a sentence of this shape:

Given these bars, at this timeframe, with these indicator settings, evaluated this way, the rule would have produced these signals.

Change any clause and the output changes. That is not a flaw in backtesting. It is what backtesting is. The discipline lies in never dropping the "given" part when you quote the result to yourself.

This is why the PocketX strategy workspace keeps the evidence attached to the saved strategy instead of reducing it to a headline figure. The evidence is the claim. A single number is not.

Read the data assumptions before the results

Before looking at any output, establish what the test was run on:

• Which instrument, and over what period. A rule tested across one trending year is a rule tested in one regime.
• Which timeframe. The same rule on daily and on five-minute bars is two different strategies with two different cost structures.
• How bars were evaluated. PocketX cash strategies evaluate closed bars. A signal is recognised at the close of the bar that satisfied the rule, not at the tick within it.
• What coverage the data has. Where history is incomplete, PocketX says so. A gap is reported as a gap, not filled in silently.

That last point deserves emphasis. Missing data quietly interpolated is how a backtest becomes fiction. If the platform tells you coverage is partial, the correct response is to narrow the claim, not to ignore the warning.

The closed-bar rule cuts both ways

Evaluating on closed bars removes a whole class of false results. A rule cannot be credited with catching an intrabar spike that reversed before the close, because the rule never saw that spike as a completed observation.

It also imposes an honest cost. The signal arrives at the close, and any action you take happens after that. A backtest built on closed bars is therefore closer to what you could actually have done — which is the only version worth measuring.

Be suspicious of any backtest anywhere that appears to enter at the exact price that triggered the condition. That is usually a lookahead artefact, and it is the single most common reason a strategy that "worked" historically fails immediately in live conditions.

What the numbers cannot tell you

Even a clean backtest is silent on several things that will decide your outcome:

• Slippage and impact. Historical bars do not know that your order would have moved the book, or that the spread widens exactly when your signal fires.
• Costs. Brokerage, exchange charges, STT and stamp duty are real, recurring, and disproportionately punishing on high-frequency rules.
• Regime change. A mean-reversion rule tested through a range-bound period is being graded on the exact conditions it was built for.
• Your behaviour. The backtest assumes the rule was followed every single time. You are the variable it cannot model.
• Survivorship in your own selection. If you tried eleven variants and kept the one that looked best, you have not found an edge. You have found the luckiest of eleven.

That last item is the one that catches experienced people. The remedy is boring and effective: decide the rule first, test it once, and treat a poor result as information rather than as a starting point for tuning.

Overfitting has a smell

You can usually detect a fitted rule without any statistics:

• It has several conditions, each of which excludes one specific historical loss.
• Its parameters are oddly precise — a 37-period EMA, an RSI threshold of 68.
• Small changes to a parameter produce large changes in the result. A genuine effect degrades gracefully; a fitted one collapses.
• The rule cannot be explained in one sentence to someone who does not trade.

The PocketX cash strategy language limits you to a flat ALL/ANY group with no nesting, precisely because nesting makes it easy to build a rule that describes the past instead of a rule that describes the market.

How to use the evidence sensibly

A workable sequence:

1. Validate the rule so you know it is well-formed and every reference resolves.
2. Backtest once, on the instrument and timeframe you actually intend to trade.
3. Read the coverage notes before the performance figures.
4. Perturb one parameter and re-run. If the result collapses, the rule was fitted to noise.
5. Save the strategy so the evidence stays attached to it.
6. Arm it and compare live signals to the historical pattern for a period before committing capital.

PocketX requires current proof before supported activation. Historical evidence still cannot guarantee future results.

Remember what arming does

Arming a cash strategy in PocketX creates alerts. It does not place orders. The backtest describes signals, and signals are what the live strategy produces — the decision to act on one is always yours, reviewed and submitted manually.

This alignment is deliberate. When the backtest measures signals and the live system produces signals, the two are comparable. Systems that backtest fills but deliver alerts are comparing different things.

A modest standard

Before you trust a rule with money, you should be able to say all of the following without flinching:

• I can state the rule in one sentence.
• I know the period, timeframe and instrument it was tested on.
• I know where the data coverage was incomplete.
• I did not tune it after seeing the result.
• I know roughly what costs will do to it.
• I know what I will do when it produces three losses in a row.

If any of those is missing, the backtest has not finished doing its job. Read how to build a rule-based strategy in PocketX for the step before this one, and how to set a stop-loss for the discipline that decides how much a wrong rule costs you.

Investments in the securities market are subject to market risk. Read all related documents carefully before investing. Nothing in this article is a recommendation to buy or sell any security, and past performance is not indicative of future results.