"Thousands of complete test runs
may be needed to evaluate rules."
We are referring to system rules here. These are the rules used for deciding
how to enter and exit the market.
System rules are not that difficult to produce. Anybody is capable of creating
new rules, especially with a little training and practice.
Of course, these rules are very important but they are by no means the whole
story. Most traders would say they're the part of the system that receives most
publicity but it's the trade management that makes the biggest difference to the
If you like, the system rules are the wide receivers but the trade management
is the quarter back.
As long as the entry and exit rules can demonstrate they have what is known as
a positive expectation, then properly managed, it should be possible to produce
But how much profit might the system be capable of making and how much risk will have to be taken to achieve it?
Those are the big questions a systems trader needs to answer when he tackles
the vital job of evaluating rules.
To make sure he leaves no stone unturned, the trader decides the full scope of
his test in advance.
Typically a time period of 5 or 10 years is chosen. Simulated live trading will
have to be run for all the commodities on the trader's list - in a mode of "separate togetherness" for each day of the period.
All of this must be done for every different combination of parameter settings
that the trader wishes to cover. This is what really multiplies the number of
complete test runs needed!
Take as an example, a set of rules that uses three different daily moving
averages. Then to test each of the averages over a range of 10 different 'day'
settings, it will require 10 times 10 times 10 complete runs.. That amounts to
1,000 runs of the whole 10 year period.
This example is by no means exceptional, so it is easy to understand why
software of high quality and speed is necessary to produce full results in the
shortest possible time.
Considering the amount of processing involved, it might sound very reasonable
if just one of the 10 year runs described were to take one minute on the
But that would mean the 1,000 run test would take 16 hours. Systems traders are
not that patient! 16 minutes would be more like what they would have in mind.
So now you understand why the evaluation process makes big demands - not on the
trader but on his software.
Copyright David Bromley 2006
All Rights Reserved.