|
I am using neural networks for classification. There are 9 independent variables out of which x7 has x1 has a correlation coeff of 0.89 and x7 and x3 has a corre coeff of 0.85. So I removed x7 and tried logistic regression and the classification accuracy jumped up considerably over the all variables model. But after removing x7 I tried out neural networks and the classification accuracy reduces as compared to the all vairables model. The obervation in case of neural networks seems to be worrying or is it right. |
|
I got the architecture by training the network by not freezing the weights and increasing the hidden neurons one at a time. The hidden neurons which gave the max classification accuracy over validation set was selected as the architecture. Acuracy was measured by the standard formula considering true negative, true positive, false postive false negative. The validation accuracy increased but the test accuracy was low as compared to the model in which all variables were considered along with its respective chosen architecture. But my basic question is if the accuracy is better for logistic regression then is the same expected from neural networks or it can be different. i.e If correlation between variables affect the output of logistic regression then the same effect should be observed in case of neural networks also. WE treat multicollinearity problem before we start logistic regression. Is the same to be followed for neural networks. Just for information both the methods(LR and NN) used the same training, validation and test data. Well, technically there isn't much difference between logistic regression and standard feed-forward back-propagating NNs (also sometimes referred to as multi-layered perceptions). If you use a sigmoid instead of a soft-max for a standard perceptron you'll notice the equations for logistic regression and the perceptron are nearly identical. The main difference being that logistic regression uses a vector whereas the perceptron a matrix.
(Dec 29 '11 at 16:01)
nop
|
|
I think you need to provide more details in the exact architecture you used for the NN, the method of training, and exactly how you measured accuracy. For example, I might see how that could occur if: (a) you had too few training samples, (b) the difference in accuracy isn't significant, (c) or due to many other factors that could be involved. |
