Hi,

Assume we have 99999 images of size 50*50. We want to learn features from this data and then do some classification. Sparse coding and convolution neural network(CNN) have been used.

Here is the scenario:

  1. Start by extracting 10 patches of size 11x11 from each of 99999 images of size 50x50. so we will have 999990 images of size 11x11.
  2. Then using [999990,11x11] let say learn 75 features.
  3. Apply window size of 11x11 for convolution operation on the 50*50 images using these 75 learned features. (we will have now 75x40x40)
  4. Apply max pooling size of 2x2( then we will have 75x20x20)

Briefly speaking after learning 75 features and applying convolution/pooling we will have 74999250 images of size 20x20(i.e [74999250,20x20]).

Now my question is that if I want to apply another layer of convolution/pooling( basically repeat all above steps 1-4), in step one for feature learning my training data set would be [74999250,20*20]?? In another words, when adding multiple layer of convolution/pooling, in fact we are increasing number input data[i.e training data] meanwhile decreasing the size of input, is it correct???

Any help appreciated.

asked Oct 11 '13 at 03:02

Inc7's gravatar image

Inc7
1112

edited Oct 15 '13 at 03:40


3 Answers:

That sounds right, so in the second round you use [74999250,11x11] images to learn another 75*X features (assuming all X features are connected to all 75 in the previous layer of your network). That sounds really costly. Appreciate an expert's opinion since I am not one.

answered Oct 16 '13 at 02:06

Ng0323's gravatar image

Ng0323
1567915

Hi lnc7 check this out. Look at figure 1 think about it and it should answer your question. Hopefully :)

answered Oct 13 '13 at 09:00

gerard's gravatar image

gerard
767912

edited Oct 13 '13 at 09:00

Looking at the vast amount of parameters, maybe it would have to apply regularization to avoid using more samples than parameters. See the slides from http://trent.st/ffx/ for an easy introduction.

answered Oct 12 '13 at 06:28

Hannes2's gravatar image

Hannes2
1

Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.