Search Algorithm

Synergy data mining application developed by AdaptiveTradingSystems.com
Post Reply
jamess
Site Admin
Posts: 55
Joined: Wed Jan 04, 2017 9:52 pm

Search Algorithm

Post by jamess » Thu Apr 13, 2017 2:42 am

There was a significant high-level change to the search algorithm in version 3.0.0 of Synergy. The following description assumes genetic optimization of the model structure is not being done and the autovalidation tests, including the walk-forward test, are being done.

1. Randomly construct a new model and randomly assign the model parameter values.

2. Measure the fitness of the model over the build period using the selected Fitness Function. The randomly assigned model parameter values are not used beyond this point. If the fitness of the model over the build period does not equal or exceed the minimum fitness level then go back to 1.

3. Apply the PSO to the model to arrive at new model parameter values.

4. Compute the fitness of the model over the build period. If the fitness of the model over the build period does not equal or exceed the minimum fitness level then go back to 1.

5. Compute the percentage of bots in the swarm that met or exceeded the minimum fitness level over the build period. If the percentage of 'fit' bots does not equal or exceed the minimum pass rate then go back to 1.

6. Compute the in-sample statistics.

7. Apply the static autovalidation tests. If any autovalidation test fails then go back to 1.

8. Apply the walk-forward test. If the model passes the walk-forward test then it is retained, otherwise go back to 1.

9. Compute the remaining statistics.

This new approach results in more robust models being found in far less time. Productivity is often somewhere between 2 to 5 times greater. The PSO is no longer optionally applied. The User Guide describes doing an exploratory modeling run by simply disabling the walk-forward test. This works reasonably well.

A wide variety in the number of models evaluated per second is expected. The model evaluation rate depends heavily on the percentage of models making through to each of the different stages. If very few models are making it past step 2. then the evaluation rate will be very high (10,000 to 17,0000 evals/sec). If a lot of models are making to the walk-forward test then the CPU threads will be busy running the walk-forward tests and the model evaluation rate will be very low. The model evaluation rate is not the important statistic, it's the number of robust models per unit of time that are produced that is important. Naturally, we need useful input data to produce any robust models.

New functionality for evaluating candidate data series is being worked on. The Data Evaluator will be very fast and will not use any out-of-sample data. When disabling the PSO for exploratory modeling runs, it was tempting to peek at the out-of-sample performance, which would often be misleading for more than one reason. For example, we might conclude that a given data series is no good for modeling our traded series because of poor out-of-sample performance when the poor performance was purely due to the random assignment of model parameter values (happen to be on a narrow peak in the model parameter versus fitness space).

Kind Regards,

James

CarlosR
Posts: 51
Joined: Sun Jan 08, 2017 10:52 pm

Re: Search Algorithm

Post by CarlosR » Thu Apr 13, 2017 12:40 pm

James, this seems like a good work flow to me. I believe the resulting quality of the models should be higher, and the improved speed in v 3.0 is quite noticeable.

mrs
Posts: 3
Joined: Thu Jan 26, 2017 10:32 am

Re: Search Algorithm

Post by mrs » Mon Apr 17, 2017 9:48 pm

I experienced a 20x speed increase on files which did not have walkforward enabled and about 1500x (!) increase when walkforward is enabled.

Mike

mrs
Posts: 3
Joined: Thu Jan 26, 2017 10:32 am

Re: Search Algorithm

Post by mrs » Mon Apr 24, 2017 2:14 pm

With this speed increase it may be feasible to discover M1 timeframe models (the lowest useful timeframe has been H1 for me). I have started testing with a large M1 time interval (about 18 months from build start to validation end).

mrs
Posts: 3
Joined: Thu Jan 26, 2017 10:32 am

Re: Search Algorithm

Post by mrs » Sun Jun 04, 2017 4:30 pm

With this new speed increase I have been able to create (potentially) profitable M1 forex models. Great work!

Post Reply