Posted By: Tom D   Date: Wed Nov 5, 2008

Title: Change

Message:
  Funny your comment about "change coming". Change in the sense of dimes and nickles is what the current model has given us. :) I have been a big fan of neural nets since I did my own nets in Santa Fe fifteen years ago with California Scientific's program. I cannot fault your math or general methodological approaches, but your frequent model failures and replacements speak volumes to inadequate data inputs. Good technique, bad data in other words, in my opinion.
 
  Now you are moving to even shorter term approaches which I think is wrong. Short term moves and data are just noise. I think the whole data set and time span needs to be re-evaluated. Maybe you need some value and economic data of key importance.

Response:
  Thanks Tom for taking the time to write.

  Let me first address the issue of updates. Updates are issued when I find a significant improvement in the ability of the model to predict the market. Updates have been appearing at a rate of about four per year. The previous update version 16.1 was used for 96 trading days and had a return of positive 11.1 percent. The current update version 16.2 has been in use for 86 trading days and has a return of negative 2.2 percent. Clearly the last few months have been difficult given the unpredictability of the financial crisis.

  When I started in this business, I thought that predicting the slope of the Dow was the right thing to do. All models up to now did exactly that. When you asked what should the forecast horizon be, the answer came back 15 days. Shorter or longer values gave a lower average accuracy of the predicted slope. The slope was analyzed by the trading model which had the task of making money. The trading model produced long, short, and cash signals. Unfortunately, a 15-day slope was not the best thing to use to make the most money. This is because, for example, you could have a positive slope with the market moving wildly between up and down days, like we have had over the last two months.

  I decided that the forecast model really needed to be focused on making money instead of predicting the slope; that is, the forecast model needed to predict the long, short, and cash signals. In version 20.1, the networks are trained to make the most profit directly without a trading model being needed to interpret the forecast. When I optimized the model, I found that a one-day forecast gave the best results. This was a surprise, to say the least.

  The neural networks that I use are not the perceptrons or recurrent networks that are commonly used to model markets. My networks have a dynamic ability to modify their internal structure to react to changes in market conditions. Because they are so highly non-linear, the usual back-propagation type methods for solving for the weights do not work. I use an evolutionary method for this purpose.

  I have looked at adding more economic data to the database, but I find that the networks tend to ignore them. I did add the CRB index this year and this was useful.

  The table below shows the data types that are currently used by the model:

  From these basic data types, 63 network inputs are computed. The model decides which ones to use in the forecast calculation.