|
Hello, I am not really a god in maths and just recently started to play with neural nets. I am trying to train a model to reconstruct its input. I know that my input is noise-free, but the reconstructions are often quite noisy. Maybe I'm totally wrong but my guess is that the noise balances out the error... But, generally, is there is a cost function that penalizes noise? |
|
You can try the Theano tutorial for denoising auto-encoders. If you're using any sensible error metric the noise should not "balance out". For example, if you're penalizing by sum of square error, and you have two variables (x1, x2), and the ideal values are (10, 10), and you get (8, 12), the error is 8 (if you're penalizing on the sum of the absolute errors, it is 4). In general, no good loss function will let you do that. Is this observed on the training inputs or on the test inputs? If it's on the training inputs you could probably use more hidden layers or less regularization; if it's on the test inputs you could regularize more your model. |
