Backtesting is just the procedure of using chronological records to experiment the act of some trading approach. Data Mining involves probing through data in order to establish patterns and find probable correlations between variables.

The Multiple Hypothesis Problem

Let’s presume that we backtest the approach against ten years of chronological marketplace data. The results are not very encouraging. Conversely, being violent and stumble traders as we are, we decide not to give up so easily. What about a ten day moving average? That might work out a little better, so let’s backtest it! We run another backtest and we find that the results still aren’t stellar, but they’re a bit better than the 20-day results. We decide to explore a little and run similar tests with 5-day and 30-day moving averages. Finally it occurs to us that we could actually just test every single moving average up to some point and see how they all perform. So we test the 2-day, 3-day, 4-day, and so on, all the way up to the 50-day moving average.

Now certainly some of these averages will perform poorly and others will perform fairly well, but there will have to be one of them which are the absolute best. For instance we may find that the 32-day moving average turned out to be the best performer during this particular ten year period. Does this mean that there is something special about the 32-day average and that we should be confident that it will perform well in the future? Unfortunately many traders assume this to be the case, and they just stop their analysis at this point, thinking that they’ve discovered something profound. They have fallen into the “Multiple Hypothesis Problem” pitfall.

Overfitting

Overfitting is a category of setback of the above problem. In overfitting we first look at the past and then build a single complex hypothesis that fits well with what happened.

Not likely, but we could always keep altering the model and taxing the approach in diverse samples (out of sample testing again) to see if our performance improves. When we stop getting performance improvements and the only thing that’s rising is the complexity of our model, then we know we’ve crossed the line into overfitting.

Conclusion

Data mining is a method to use our chronological price data to propose a feasible trading strategy, but that we have to be aware of the pitfalls of the multiple hypothesis problems and overfitting. The way to make sure that we don’t fall prey to these pitfalls is to backtest our strategy using a different dataset than the one we used during our data mining exploration. We commonly refer to this as “out of sample testing”.

back test moving average (1), data mining backtesting (1)

Open trade is a fantastic opportunity for the smart trader who knows how to identify supply and demand. Most of the time, our entry is within seconds to minutes of the opening bell. Prices gap up because there are more buy orders at the open morning than there is available supply at the prior day’s closing price. They gap down because there are more sell orders at the open than willing demand at the prior day’s close. Therefore, market prices are almost always at price levels where there is a supply and demand imbalance at the open in the morning trade technique.

The input is to not stare at candles on your screen as red and green pictures and patterns. You must recognize what is happening behind the scenes in the morning trade. Whether you’re trading stocks, futures, options, or forex, the logic and rules never change. Again, the market imbalances are greatest at or near the open of trading in all markets. By the end of the first hour of trading each day, a large amount of novice trading capital is simply transferred into the accounts of the astute trader. If you can’t see the novice trader in markets, you most likely are the novice trader.

The dramatic price decline can only happen because there is much more supply at that level than willing demand. The dramatic rate of decline suggests a strong supply and demand imbalance at that level. Now, notice what happens the morning of the upgrade. Our job is to find this novice trader and simply take the other side of his or her trade and make the best of morning trade.

When you enter markets at price levels where supply and demand are out of balance in a big way, especially at or near the open of trading, moves in the market are typically very fast. This trade and the thoughts and rules that went with it are not meant to impress you. I mean to impress upon you the importance of looking at markets for what they really are which is simply an ongoing supply and demand equation. Opportunity exists when this simple and straightforward relationship is out of balance. Everything else in and around markets is just noise that is meant to invite you into markets at the wrong time and in the wrong direction.