|
When error is back propagated in a convolution layer in CNN, do we need to average the error deltas ? As a 5x5 filter mask may traverse a pixel more than once (except some of the boundary pixels). |
|
No, they are added together. Jake Bouvrie from MIT has some good notes for implementing CNNs. It's targeted at matlab but shouldn't be too hard to adapt to other languages since the descriptions are quite thorough. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.70.1419&rep=rep1&type=pdf |
I am using stochastic gradient method (no batch processing).
pushing error deltas from a layer with feature maps of size 9x9 to a layer with features maps of size 13x13, the error deltas may be computed following number of times
count13by13[169] ={1,2,3,4,5,5,5,5,5,4,3,2,1, 2,4,6,8,10,10,10,10,10,8,6,4,2, 3,6,9,12,15,15,15,15,15,12,9,6,3, 4,8,12,16,20,20,20,20,20,16,12,8,4, 5,10,15,20,25,25,25,25,25,20,15,10,5, 5,10,15,20,25,25,25,25,25,20,15,10,5, 5,10,15,20,25,25,25,25,25,20,15,10,5, 5,10,15,20,25,25,25,25,25,20,15,10,5, 5,10,15,20,25,25,25,25,25,20,15,10,5, 4,8,12,16,20,20,20,20,20,16,12,8,4, 3,6,9,12,15,15,15,15,15,12,9,6,3, 2,4,6,8,10,10,10,10,10,8,6,4,2, 1,2,3,4,5,5,5,5,5,4,3,2,1};
do we need to average the error gradient accordingly ?