MarketTrak Question/Comment Message

Posted By: Jim   Date: Sat Aug 25, 2007

Title: LargeSampleSize? CurveFitting? Robust?

  I get very concerned when I see such great performance improvement. Such an improvement, old Trading Model return +20.0% vs new Trading Model return of +39.5%, can be the result of using(1)too small a sample size covering a limited market period, and(2)too complex,too many parameters used to fit to old data, Curve Fitting. If true, it will not represent what we see going forward under varying market conditions -not Robust. Can you please assure me that you used (1) large enough sample size to cover all types of markets,(2) not overly complex parameters so that when the underlying market data changes, the model will hold up? My concern is that that your perfomance will not hold up going forward. Can you please comment...Thanks, Jim

  You raise very valid concerns. I never know for sure how a model will behave in the future. But here are some facts that may make you feel better about the update: 1) The training set was increased to 4700 days; previously it was 4500 days. 2) The model has about 45 free parameters so there is about 100 days of data for each one. I don't think there is an overfitting problem. 3) The last 200 days of data were not used training the networks. The results you see are truly blind.

  Time will tell if this model is going to do well. The update is fairly complex and large in size. The next update, currently under development, is even more so.

Return to message board

Bag Toss: high quality, made in America, and makes a great gift.

Copyright © 1996-2017 MarketTrak LLC.  All Rights Reserved.