Wish List

Synergy data mining application developed by AdaptiveTradingSystems.com
CarlosR
Posts: 51
Joined: Sun Jan 08, 2017 10:52 pm

Re: Wish List

Post by CarlosR » Mon Jul 17, 2017 11:27 pm

Hi James,

Would it be possible to add some way to save the Data Snoop results? If the data covers a lot of years, the runs can take a lot of time, and it would be nice to be able to save them. I find that often times the highest rated input is not the best one for my model building purposes, and I have to keep going back and trying others. Right now I'm saving a screen shot of the results, but that only gets the top ones. ( at least without making the process real cumbersome)

Anything you can do along these lines would be appreciated.

jamess
Site Admin
Posts: 55
Joined: Wed Jan 04, 2017 9:52 pm

Re: Wish List

Post by jamess » Mon Jul 17, 2017 11:39 pm

Hi Carlos,

I'll add functionality to save and load data snoop runs.

Kind Regards,

James

CarlosR
Posts: 51
Joined: Sun Jan 08, 2017 10:52 pm

Re: Wish List

Post by CarlosR » Thu Jul 20, 2017 11:25 pm

James, on the subject of Data Snoop, would it be possible to allow the user to determine the relative weights used to calculate the Rating? For that matter, how is the rating calculated now? It's not intuitively obvious -- I see some inputs listed in the Evaluator that have nice specs but low ratings.

Thanks for any light you can shed on this.

Schutten
Posts: 7
Joined: Thu Jan 12, 2017 11:02 am

Re: Wish List

Post by Schutten » Tue Jul 25, 2017 3:53 pm

Its good that new versions are coming with new features. However I need to run the conversion tool quite alot. Most important runs I upload to the database. Would it be possible for you James to do a conversion run on the database once in a while?

Regards,
Dennis

jamess
Site Admin
Posts: 55
Joined: Wed Jan 04, 2017 9:52 pm

Re: Wish List

Post by jamess » Tue Jul 25, 2017 11:10 pm

Hi Dennis,

I will manually bring the modeling runs in the database up to date today.

Kind Regards,

James

jamess
Site Admin
Posts: 55
Joined: Wed Jan 04, 2017 9:52 pm

Re: Wish List

Post by jamess » Tue Jul 25, 2017 11:33 pm

Hi Carlos,

Regarding the Data Snoop rating, It is the proportion of randomly constructed models that met the minimum fitness requirements over the build period and the validation period. Only models that initially meet the minimum fitness requirements over the build period are counted and corresponds to the number of samples.

1. A model is randomly constructed with randomly assigned model parameter values.

2. The percent of perfect, equity straightness and percent winners produced by the model signal are measured over the build period. If the fitness levels do not meet the minimum requirements then back to step 1. If the model meets the minimum fitness requirements then it is counted as a sample.

3. The percent of perfect, equity straightness and percent winners over the validation period are then measured. If the model fails to meet the minimum fitness requirements over the validation period then the model counts as a failed sample, otherwise it counts as a successful sample.

The rating is the percent of samples that are successful. When there are less than 20 samples, the rating is diminished by design. As the Data Snoop gathers more samples, the confidence in the rating goes up.

To understand the motivation for this approach it's helpful to imagine what you would expect if the input data was not useful for modeling the traded series or that the traded series is completely random. There would be some models that happen to work over the build period, but the chance of them continuing to perform well up to the end of the validation period is slim. Remember we start by only considering models that happen to work over the build period, otherwise you might end up thinking 'why not just measure performance over the validation period initially'.

When the input data is useful for modeling the traded series, the proportion of models that happen to perform over the build period and then continue to perform up to the end of the validation period increases. Some of the randomly constructed models actually model a meaningful relationship.

In the user guide I suggest that a relatively high number of samples and a high rating is desirable. A relatively high number of samples indicates that it is relatively easy to build models that meet the minimum fitness requirements. There are a few things that can be done to improve the algorithm. It's a good start for now though.

Kind Regards,

James

CarlosR
Posts: 51
Joined: Sun Jan 08, 2017 10:52 pm

Re: Wish List

Post by CarlosR » Wed Jul 26, 2017 12:12 am

Hi James,

Thanks very much for the detailed reply. There's a lot of information in your post for us to digest!

Let me ponder it a bit, and I may have some more questions. I think it's an important subject for all users to understand.

Post Reply