If you have account already, Log in here first

Log In
  • en
    • cs
    • pl

Automated trading 8: Why just a backtest is not enough?

28 Nov 2017,

Backtests simply simulate our strategies on historical data. You can then see how the strategy would behave a month ago, a year ago, or a hundred years ago.

Having a good backtest is important. It will help us understand what we can expect from the strategy, whether it is profitable at all, how much it can potentially earn, what are the risks.

Backtest itself is not enough. This is just the beginning and we must work with it. Beginners often focus only on the backtests, then they deploy the strategy and let it work. This leads to prejudices that automated strategies do not work.

Let’s have a look on what must be done with the backtest in the next steps.

I have three basic requirements when building strategies:

  1. Backtesting
  2. Testing
  3. Optimization / Verification

Backtest is done on high-quality data, as we have said last time. Today we are going to talk about testing.

I use several metrics to test my strategy. By doing so, I find out how the strategy is sensitive to market changes, brokerage, etc. I call this analysis test of quality, or robustness test. Let’s look at what it is about.

Testing quality of the strategy – example

Let’s take a simple strategy as an example. We are entering the trade when the moving average with period 50 rises above the moving average with period 200. Stop loss and profit target are 200 pips. This strategy is just an example, do not try to trade it, please.

However, it could happen that its parameters, the moving averages and the SL and PT, would be “squeezed” to historical data and the strategy would be profitable on the backtest. So we need to measure how sensitive is the strategy to its parameters.

Backtest can look like this:

For testing, we use so-called Monte Carlo simulation. It sounds complicated, but it’s easy. For each strategy, we will perform 100 to 200 backtests, each with random parameters within a certain range. So, in our case, we would choose the moving averages in the range from 70 to 130. Respectively 140 to 260 for the second moving average, SL and PT would be then between 140 and 260 pips. From the outcome we calculate the results we can expect. Of course, we do not need to do the whole test manually, but we can do it automatically, this feature is built-in in the StrategyQuant.

The result then looks like this:

On the right, we can see all 100 simulations and in the left part there is the statistics. We are interested in the grey line and we can see that the profit has fallen by about half and the DD (risk) is twice higher. The result is definitely not ideal and I would not use such strategy in real trading.

I do more of such tests:

  • Changing the order of the trades
  • Skipping trades
  • Change in market volatility
  • Etc…


Building quality strategies is primarily about testing. Basically, no matter how we build our strategies, we must always do these tests. It is done even by a colleague who uses artificial intelligence to analyse the markets. In short, quality testing is the basis. It may seem complicated at first glance, but it can be also fun. You do not find these tools in Metatrader, so I’m working with StrategyQuant, where all of these tools are incorporated.

Option report

Get your report from Jan Kaska's trading every week!


Leave a Reply

Your email address will not be published. Required fields are marked *