I am trying to train a genetic algorithm trained single layer feed forward artificial neural network.

I am using the Machine CPU data set from this link from this page. I have scaled both the inputs and outputs of the dataset using normalizeData(mcpu, "0_1") function in R.

I have installed the ANN package available in R, and using the ANNGA function to train a single hidden layer network with varying hidden layer units. The train set is the first 100 examples in the scaled dataset, and the remaining next 109 examples are used to test the trained model.

I am using RMSE measure. (Note: ANNGA gives MSE).

My question is that, the RMSE for the train set is good and compares to Steepest Descent Backpropagation and QuickPropagation and another population based algorithm on which I am working, but the model trained by ANNGA gives poor test error. I have tried different ranges of max population, number of generations, mutation rate, crossover rate and min and max weight values. I have computed the RMSE 10 times for each configuration and took the average, min, max of the results and displayed to inspect.

My question is, am I doing it right? And, why I am not getting good test error with this ANN package?

I am expecting RMSE around 0.065 ~ 0.07 and what I get is 0.1 ~ 0.11 .

Note: ANN has sigmoid threshold, and the other methods I have used also use sigmoid threshold and all data are scaled within 0 and 1.

Please let me know if you want to see the code (in R).

asked Apr 12 '13 at 15:39

phoxis's gravatar image

phoxis
317711

Why would you want to do this?

(Apr 12 '13 at 23:33) gdahl ♦

To compare errors with other population based algorithms.

(Apr 13 '13 at 04:04) phoxis

One Answer:

Agree with gdahl on this -- genetic algorithms are probably not a good optimization choice for neural nets. It is much better to use the gradient information (e.g. with stochastic gradient descent). That's also quite a small training set. It sounds like an overfitting problem, and for whatever reason genetic algorithm optimization may be more prone to overfitting. You could try increasing the regularization (weight norm penalty) and see if that helps.

answered Apr 13 '13 at 17:07

Andrew%20Maas's gravatar image

Andrew Maas
16113

Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.