Backtesting is just the procedure of using chronological records to experiment the act of some trading approach. Data Mining involves probing through data in order to establish patterns and find probable correlations between variables.

The Multiple Hypothesis Problem

Let’s presume that we backtest the approach against ten years of chronological marketplace data. The results are not very encouraging. Conversely, being violent and stumble traders as we are, we decide not to give up so easily. What about a ten day moving average? That might work out a little better, so let’s backtest it! We run another backtest and we find that the results still aren’t stellar, but they’re a bit better than the 20-day results. We decide to explore a little and run similar tests with 5-day and 30-day moving averages. Finally it occurs to us that we could actually just test every single moving average up to some point and see how they all perform. So we test the 2-day, 3-day, 4-day, and so on, all the way up to the 50-day moving average.

Now certainly some of these averages will perform poorly and others will perform fairly well, but there will have to be one of them which are the absolute best. For instance we may find that the 32-day moving average turned out to be the best performer during this particular ten year period. Does this mean that there is something special about the 32-day average and that we should be confident that it will perform well in the future? Unfortunately many traders assume this to be the case, and they just stop their analysis at this point, thinking that they’ve discovered something profound. They have fallen into the “Multiple Hypothesis Problem” pitfall.

Overfitting

Overfitting is a category of setback of the above problem. In overfitting we first look at the past and then build a single complex hypothesis that fits well with what happened.

Not likely, but we could always keep altering the model and taxing the approach in diverse samples (out of sample testing again) to see if our performance improves. When we stop getting performance improvements and the only thing that’s rising is the complexity of our model, then we know we’ve crossed the line into overfitting.

Conclusion

Data mining is a method to use our chronological price data to propose a feasible trading strategy, but that we have to be aware of the pitfalls of the multiple hypothesis problems and overfitting. The way to make sure that we don’t fall prey to these pitfalls is to backtest our strategy using a different dataset than the one we used during our data mining exploration. We commonly refer to this as “out of sample testing”.

backtesting overfitting (3), backtesting pitfalls (2), back test moving average (1), backtesting data mining strategy pitfall (1), data mining backtesting (1), data mining problem trading hypothesis (1), multiple hypothesis testing stock market (1), multiple hypothesis testing stockmarket (1), trading data-mining (1)

Trade price analysis is essentially a top down approach, working from the macro level of Market Structure such that we analyze the big picture first and then go down to the current trend within that structure. Finally take a look at the current price pattern through candlestick analysis or other method that works.

Market structure

The higher timeframe chart is opened and any areas of major support or resistance are identified. Support & Resistance are areas of past price congestion. Trade is a higher probability of price stalling or reversing at these areas of major support or resistance. Then narrow focus to the shorter trading timeframe and add to the market structure framework, by identifying areas of minor support or resistance. Thus Market Structure is simply identifying a support and resistance framework within which price moves.

Having defined the market structure an analysis on the trend to identify its strength is conducted. If the trend moves strongly, anticipate it being more likely to break through the next support or resistance levels. If it is weakening, a greater probability of the support or resistance levels forming a barrier to further price movement can be declared. We determine the strength of the trend by looking at its closeness to the support and resistance barriers within the framework.

Conduct further price analysis regarding the trend and how it moves within the support and resistance framework. The price may have just meandered slowly up to a major resistance level. The current price swing may clearly show less momentum than both the previous upswing and downswing. And the price bar range may be narrowing. This gives a reduced likelihood of the commitment required from the bulls to break through the area of increased supply. The shooting star pattern provides evidence of a clear rejection of prices at that resistance level. This provides a lower risk or higher probability trade in the short direction.

Instead of entering long on a cross reversal pattern, just because it matches the cross on candlestick patterns, conduct further analysis to see where this pattern occurs within the bigger picture of market structure. The trend may show a strong and accelerating move downward, on greatly increased volume, extending price rapidly to great distances below its average. This is an area where I expect increased demand. The cross shows a clear halting of the rapid move down, and allows me an opportunity to enter a low risk trade close to an area of major price support.

The end result might be the same. Over a lifetime of trading this approach will produce more favorable results than just entering because the pattern matched.

The market structure defines where we trade. The trigger, whether a candlestick pattern or some other form of entry trigger, tells you when to get in, only when you’ve first met the requirements of the market structure rule.

Think about where the current price movement is within a structure of support and resistance. Think about the changing strength of the current trend, or price swing, as it approaches this area of support or resistance. Watch for signs of strength or weakness in the trend, through the clues obvious in changes of drive and instability.

And don’t forget – always use stops, because there are no guarantees. This is a game of chance.

‘Buyer perception’ varies among businessmen. It is the potential buyers do not buy the vehicle because they perceive a commodity as not unworthy it for them at the asking cost. Of course, they may think that the asking price is fair, but they didn’t like the color or the make. Yet, those prospective buyers may have bought the car anyway had the price been much lower. For those prospective buyers that did not buy, there may be a price that would turn them into buyers.

Presume you had ten potential car buyers out looking at the same car. At the present asking price, maybe only one of the ten feels it is worth buying at the asking price and would buy it. As you keep lowering the price, however, more and more may then consider it a good buy and want to buy it. Perception is a mental function. Therefore, we must address the ‘mental’, the ‘psychological’ angle of buying. When we consider the psychological aspect, we must consider the ‘emotional’ aspect as well. Our mental perception is mostly affected by information.

Working back from the actual buying and selling to the effects that may show the way to those final actions, you get a better accepting of the inner workings of market price action. The market trader can also reach a point of making highly accurate forecasts of market behavior, enough so as to be able to increase ones odds of getting on the right side of most trades.

Open trade is a fantastic opportunity for the smart trader who knows how to identify supply and demand. Most of the time, our entry is within seconds to minutes of the opening bell. Prices gap up because there are more buy orders at the open morning than there is available supply at the prior day’s closing price. They gap down because there are more sell orders at the open than willing demand at the prior day’s close. Therefore, market prices are almost always at price levels where there is a supply and demand imbalance at the open in the morning trade technique.

The input is to not stare at candles on your screen as red and green pictures and patterns. You must recognize what is happening behind the scenes in the morning trade. Whether you’re trading stocks, futures, options, or forex, the logic and rules never change. Again, the market imbalances are greatest at or near the open of trading in all markets. By the end of the first hour of trading each day, a large amount of novice trading capital is simply transferred into the accounts of the astute trader. If you can’t see the novice trader in markets, you most likely are the novice trader.

The dramatic price decline can only happen because there is much more supply at that level than willing demand. The dramatic rate of decline suggests a strong supply and demand imbalance at that level. Now, notice what happens the morning of the upgrade. Our job is to find this novice trader and simply take the other side of his or her trade and make the best of morning trade.

When you enter markets at price levels where supply and demand are out of balance in a big way, especially at or near the open of trading, moves in the market are typically very fast. This trade and the thoughts and rules that went with it are not meant to impress you. I mean to impress upon you the importance of looking at markets for what they really are which is simply an ongoing supply and demand equation. Opportunity exists when this simple and straightforward relationship is out of balance. Everything else in and around markets is just noise that is meant to invite you into markets at the wrong time and in the wrong direction.