Dropping 0.61% of the data destroyed 96% of the profit

A backtest of mine reported about ₹3.53M of profit. It was not a silly backtest. The logic was reasonable, the costs went through a validated library, the code did what it was supposed to.

Then I applied three data-integrity rules. They dropped 84 symbol-days out of 13,732 — 0.61% of the data. Roughly 96% of the profit disappeared with them.

The strategy had been trading almost entirely on broken prices.

The three rules

They were written before I looked at what they would remove, which matters — a cleaning rule invented after seeing the results is just a way of choosing your answer.

One: drop the day if the mid price escapes the feed's own circuit band. Exchanges publish upper and lower circuit limits per stock per day. A price outside that band is not a price. It is a data error, and the feed itself tells you the bounds.

Two: drop the day if coverage is bad. Specifically, a gap of more than 30 minutes, or fewer than 200 bars with actual ticks before the 15:15 square-off. A day with holes is a day where your backtest fills gaps with assumptions and then reports the assumptions as profit.

Three: drop the day if any one-minute mid move exceeds 800 basis points. An 8% move inside a single minute on a liquid name is almost always a corporate action or a bad print, not a trade you could have made.

What they caught

The clearest catch was TATAMOTORS, 14 October 2025: a price recorded as ₹991 becoming ₹99.

That is a stock split. The company divides each share into ten, so the price divides by ten. Every holder is exactly as wealthy as before. Nothing happened economically.

To a backtester reading raw prices, a 90% overnight crash happened. And if the strategy has any mean-reversion or dip-buying instinct, it just found the greatest opportunity in the dataset. It buys with everything it has, the price "recovers" — meaning the data goes back to normal on some later day, or the strategy exits against a price on the pre-split scale — and it books an enormous fictional profit.

The other catches had the same character. A Muhurat session — the ceremonial hour-long trading session on Diwali — which is a real session with real prints, but its shape is nothing like a normal day, and 200 bars of coverage do not exist. Two more single-day price anomalies on other names.

None of these are bugs in the strategy. The code was correct throughout. The code was correctly processing prices that did not mean what the code assumed they meant.

Why this ratio is the whole point

The number I keep returning to is the ratio: 0.61% of the data carried 96% of the profit.

That asymmetry is not a coincidence, and it is not specific to my data. It is what you should expect, for a reason worth stating plainly.

A backtest is an optimiser. Not formally — but it searches over parameters and picks what performed best, and that search is drawn to the largest returns in the sample. Broken data produces the largest returns in the sample, because a 90% price move dwarfs anything the market actually did. So a corrupted day is not diluted by the 99.4% of clean data around it. It dominates, precisely because it is extreme.

This means the usual intuition — a bit of dirty data adds a bit of noise — is exactly backwards. Dirty data does not add noise proportional to its frequency. It adds bias proportional to its extremity, and the extremity is why it survived into your results.

Which leads to the practical rule I now follow: a data-integrity layer is not optional hygiene to do when there is time. It is the first filter, before any signal work. It is also the cheapest thing in my whole research stack — three rules, a few dozen lines, no modelling, no tuning — and the most transferable, because it depends on the feed rather than the strategy. Every strategy I test now sits behind it.

Four other ways correct code produces a wrong answer

Merging the folders surfaced a family of related failures. What unites them is that in every case the code ran without error and produced a plausible number.

The universe you selected on. A breakout configuration was best on a 400-symbol training sample, at +₹17,201. Re-scored on the full 1,091-symbol universe, identical logic, it went negative — both net and gross — in both windows. Expanding the symbol set, changing nothing else, reversed the sign. The configuration had not found a pattern. It had found the 400 symbols. This is cheap to check and I now treat it as mandatory: a configuration selected on a subset must be re-scored on the full universe before I believe it.

A metric that fires on pure noise. I used pooled information coefficient as a screen for a while. Then I ran it on permuted data — same values, shuffled so any real relationship is destroyed — and it manufactured t-statistics up to 4.9. Meanwhile, on real data, the configuration that actually netted the most money carried a negative pooled IC. The metric and the outcome had come apart completely. A t-statistic near 5 that a permutation null also produces is not evidence of anything. The correct procedure is the one another folder used: it produced in-sample |t| up to 5.02 across 40 symbols and then reported 50.3% held-out sign accuracy — chance — and rejected the idea on the split rather than on the t-statistic.

A benchmark that harvests noise. I built a foresight benchmark — the best achievable profit with perfect knowledge, subject to my real position limits — to measure what fraction of the available money a strategy captures. It yields ₹392,748 a day on real data. On martingale surrogates, which are random walks with no predictable structure whatsoever, it yields ₹424,226 a day. More. The benchmark was measuring how much a price path wanders, not how much of it is predictable. Capture ratio against it is a progress metric at best, and never evidence of edge.

Regime drift wearing a model's clothes. A direction model reported AUC 0.53, modestly better than chance. Day-neutralising it — removing cross-day drift so the model has to discriminate within each day rather than across days — dropped it to 0.506. Chance. It had learned that some days trend up and some trend down, which is true, and useless, because you have to trade inside a day without knowing which kind it is.

The unit errors that manufacture nulls

There is a mirror-image failure that I think gets far less attention than it deserves, because its output is a negative result and negative results do not attract scrutiny.

A cost library took quantity in lots. A folder passed quantity=1, correctly documented as lots — and then computed gross P&L as though it had traded one share. Against a flat brokerage cap, that is a trade whose costs are enormous relative to a one-share position. Every result comes out negative.

That bug manufactures nulls. It cannot produce a false positive. It can only kill ideas that might have worked.

I found this trap twice across the folders. One session caught it in a predecessor's code; another still had it. Which means an unknown number of "this does not work" conclusions in my own archive may be arithmetic rather than findings.

A related one: a cost call wrapped in a bare try/except, silently falling back to a hand-estimate of 0.1% of notional with no log line. An unknown subset of that folder's "net" figures never touched the cost library at all. There is no way to tell from the output which ones.

The general lesson has changed how I read my own negative results. A null is a claim and it needs defending like any other. Before I accept "this does not work," I now check the specific errors that produce exactly that outcome: units and lots, a cost path that silently fell back, a universe filter that dropped the names the signal was about, a label misaligned against the feature. A null I have not tried to break is not a finding. It is an untested hypothesis with a discouraging shape.

Where the honesty has to sit

I should be clear about the status of all this.

The data-integrity rules are the strongest thing here. They are pre-registered, the catches are named and verifiable, and the mechanism is obvious once stated. I would defend that one.

The rest are single measurements from single folders, mostly not independently re-run. The 96% figure comes from one uncontrolled backtest, and a different strategy on the same feed would lose a different fraction. The point is not the number. The point is that the number can be that large at all, from a fraction of a percent of the data.

And there is a limit I cannot argue past. When I checked whether reported figures had supporting artifacts on disk, five of fourteen folders cited results directories that no longer existed. Scratch directories had not survived the copying. Methodological quality did not predict which — one folder used the cost library correctly, documented everything, and still had nothing to verify against.

So about a third of these numbers are reproducible in principle and not citable in practice. I have written them down at the grade the evidence supports, which for several is low, rather than rounding up because they are interesting.

The reason I would still build the integrity layer first, before any of the modelling: it is the only item on this list that is cheap, mechanical, strategy-independent, and impossible to fool yourself with. Everything else here is a lesson about how a specific number misled me. The three rules are a permanent reduction in how often that can happen again.