Adaptive in the past?

Synergy data mining application developed by
Post Reply
Posts: 7
Joined: Thu Jan 12, 2017 11:02 am

Adaptive in the past?

Post by Schutten » Thu Feb 16, 2017 4:39 pm


What I have noticed is that swarms are adapting nicely in realtime trading forward however when trading on unseen data before the IS period it is failing. What should we think about this? What is your opinion about this James?



Site Admin
Posts: 55
Joined: Wed Jan 04, 2017 9:52 pm

Re: Adaptive in the past?

Post by jamess » Thu Feb 16, 2017 11:22 pm

Hi Dennis,

A couple of quick points:

1. Dakota 3 will not start adapting the model parameter values until it has enough historical performance. The number of rows that no adaptation will occur for corresponds to the length of the Lookback Period of the Bot Performance add-in in Dakota 3. If you were working with daily data and the lookback period was set to 500 then there would be no parameter adaptation for approximately 2 years.

2. I find that Dakota 3 requires the model parameter ranges to be approximately centred around the near optimal values. The range can be fairly wide, say from 8 to 16 for the period of a moving average, but cannot be extremely wide. e.g. from 2 to 100. Keeping that in mind, I wonder what would happen if the model was re-fitted to the data before the in-sample period.

I have been using Dakota 3 recently to implement models for gold futures on the daily time frame. In regards to performance before the in-sample period, I found a mix of results. Some models performed well and others chopped sideways. Ideally all models would perform well on the older data. However, I don't really know how they performed on the older data because of point 1. No adaptation was occurring for anywhere from 1 year to 4 years, depending on how long I set the lookback period of the bot performance add-in.

I think of markets as going through various regimes or modes of behaviour over time. There are many market regimes. Some are short-lived and some persist for decades. I think of the market regimes as over-lapping and stacked upon one another with the longer-term regimes at the bottom. If we use less data for model building then the resulting models will tend to reflect the current shorter-lived market regimes that are in place. As a result, they may not last very long in terms of performance. However, performance will probably be higher compared to the models built using a lot more historical records.

Sometimes you can see switches in market regimes very clearly on price charts. This is especially true for stock price histories. I've seen a stock price go from being extremely mean reverting to the opposite. It was extremely choppy for years and then suddenly, started to move in long persistent price trends. The 'choppiness' disappeared overnight. If your models were relying on the extreme mean reversion then they would have suddenly failed and continued to perform badly.

If a model is not working on older data then you might consider if there has been a switch in the dominant market regimes over the time period that you are modeling. If so then I'd be wondering how risky is it to use the models moving forward.

Kind Regards,


Posts: 51
Joined: Sun Jan 08, 2017 10:52 pm

Re: Adaptive in the past?

Post by CarlosR » Sun Feb 19, 2017 2:29 pm

Some very nice insights there, James. Thanks for sharing them.

Post Reply