Instantaneous win-rate and curve-fitting

If you flip a coin and it lands HHHHHHHHHT, we’d say that that was pretty special, but if it alternates exactly HTHTHTHTHT, then that’s OK and we don’t question it. In fact, both outcomes have the same probability.

The same is true regarding the random distribution of wins and losses that are part of whatever system you trade.
Let me illustrate with an example (and the reason behind this piece).

We took our first Samurai Scrooge loss in ages during the early hours of trade this morning

The numbers are:
1 June 2017 to 23 August:-
Total trades: 14
Winners: 13
Losers: 1

If you were to look at these figures in isolation, you’d calculate a win-rate of 92.9%.
What we are really looking at is called is instantaneous win-rate, and the potential is produces to fall into the curve-fitting trap.

Instantaneous win-rate

The long-term win-rate for the Samurai Scrooge ranges from 65%-71%. This is a metric measured over 334 trades and counting, over several years. In testing, over decades, the long-term win-rate also fluctuates over this range.

So how do we all of a sudden enjoy a 93% win-rate? Did the system change? Did the market change? No, and no.

All we are seeing is a small dataset, that, if taken in isolation, will skew our perception. Said another way, we’re just looking at a random distribution of wins and losses (HHHHHHHT) that look really good.

To show you how much your instantaneous win-rate can vary normally, have a look at this spreadsheet. It simulates a system with a 70% win-rate, which is the long-term average over 1000 trades.

But have a look at row 45. You’ll notice that the win-rate after 44 hard fought trades is sitting at just 59%. This win-rate variation has nothing to do with your system, or the market. It’s a mathematical certainty. So, can your 70% win-rate system be returning a 59% win-rate over a whole bunch of trades? You bet.

We’re enjoying the other side of the coin (excuse the pun) right now.

Curve-fitting

In trading, curve-fitting is the process where you take a little bit of data and draw conclusions about how things will be over a lot of data. In this example, if we just took our last 14 trades, we would say that we had a 93% win-rate system, which is complete and utter nonsense.
Why does this matter? Well, how much you risk on each trade will be influenced by how often you expect to win. So by curve-fitting, you will not only believe you’re going to make a lot more than you will, but you’ll risk more doing it.

Leave a Reply