Calibration and the hash-chained ledger

Every trading service on the internet shows you its winners. The mechanism is not usually fraud, it is selection. Screenshots are chosen after the fact. A published track record is a list that someone controls, and anything a publisher controls after the outcome is known carries no information about skill.

There is a second, subtler version of the same problem. A service says "high probability setup", the trade loses, and the service points out that high probability was never a guarantee. That statement is unfalsifiable in exactly the way that makes it worthless. A probability claim only means something if you can count.

Indikora addresses those two problems with two separate mechanisms, and it is important that they are separate, because they prove different things.

What calibration actually measures

Calibration is the comparison between stated confidence and observed frequency. Take every signal published at 60%, count them, and count how many resolved the way they were stated. If roughly 60 out of 100 did, the number was honest. If 30 did, it was not, regardless of whether any individual call felt reasonable.

This is a claim about the label, not about profitability. A perfectly calibrated system can lose money and a badly calibrated one can make it, because outcome size is not part of the measurement. Calibration tells you that when the engine says 60% you can take the 60% literally. What you do with a literal 60% is a separate question involving position size and the distribution of wins and losses.

The measurement has requirements that are easy to skip and fatal to skip.

The bucket has to be defined in advance. Grouping outcomes into confidence bands after seeing them is how you manufacture good calibration.

The sample has to be large enough per bucket. Twelve signals in the 70% bucket tells you nothing. A single unlucky run moves that number by 20 points.

The resolution rule has to be fixed. When a signal counts as right or wrong must be stated before publication, not decided per trade.

Calibration is usually shown as a chart with stated probability on one axis and observed frequency on the other, and a diagonal representing perfect agreement. Deviation from that diagonal is the finding. Persistent deviation in one direction means the system is systematically over or under confident, which is a measurable defect rather than a matter of opinion.

Why the ledger is hashed and chained

Calibration only works if the record of what was said cannot change. That is what the hash chain is for.

Each published signal is hashed with SHA-256, a function that turns any input into a fixed-length fingerprint. Change one character of the input and the fingerprint changes completely, in a way nobody can steer. Each new signal's hash includes the previous signal's hash, which links them into a chain.

The consequence: editing a signal from three months ago would change its hash, which would break the link to the next one, which breaks every link after it. Rewriting history means rewriting the entire chain from that point forward, and anyone who recorded a later hash can see it no longer matches.

That is a strong guarantee about one narrow thing. The chain proves the record was not edited or back-dated. It proves nothing about whether the signals were any good. A perfectly chained record of consistently wrong calls is entirely possible, and the chain would verify cleanly. Tamper-evidence and quality are different properties, and conflating them is the mistake the mechanism is most likely to invite.

How to read a backtest without being fooled

Backtests appear anywhere rules exist, and they are useful for a narrower purpose than most people assume.

A backtest measures how a fixed set of rules would have behaved on a specific historical window. Good ones tell you the shape of that behavior: how often entries fired, how long positions were held, how deep the drawdowns ran, how much of the result came from a small number of trades.

The limits are structural and do not go away with better data.

The rules were chosen by someone who had already seen the data. Even with discipline, parameter choices absorb information from the sample.

The number of independent market cycles is small. Twenty years of daily data is not twenty thousand independent observations, it is a handful of regimes.

Fills, spreads, funding and slippage are modeled, not observed. Those assumptions move results more than most people expect, especially on volatile assets.

The honest way to use a backtest is as a behavioral description, not a forecast. It answers "what does this rule do in a market like the one it was tested on" and it does not answer "what will this earn".

What to ask of any track record

Three questions that work on anything, including this one.

Was the claim recorded before the outcome? If not, it is a selection, not a record.

Can I count the claims and check the rate? If the confidence language is not numeric, you cannot.

What exactly does the verification prove? Tamper-evidence, sample size, and quality are three different things, and a service that lets you check the first two has told you something real without having claimed the third.

Verifiability is not the same as being right. It is the precondition for finding out.