Training a neural network, data set is divided randomly in train, validation and test set. This is repeated k times with different folds.
e.g. Divide into S1,S2,S3,S4,S5
trial 1- training set = S1+S2+S3, validation set = S4, test set = S5
trial 2- training set = S2+S3+S4, validation set = S5, test set = S1
and so on for total 5 trials.
Validation set used for early stopping. Test error obtained when validation error is minimum. How do I report the test performance (ie measure the generalization error)?
1. average test errors of the 5 trials
2. lowest test error from among the 5 trials
or 3. test error of the trial with lowest validation error

asked Jun 05 '14 at 02:47

Ng0323's gravatar image

Ng0323
1567915

edited Jun 05 '14 at 05:50

Be the first one to answer this question!
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.