I'm a graduate student who is implementing a CDBN for face recognition task.
Well, I have some problems with my implementation.
The first thing is understanding the data pre-processing.
After a long investigation I found out that, for large data (150pix*150pix) or even more,
1/f whitening if preferred, and local contrast normalization should be followed. (Bruno Olshausen et al.)
Well, I'm not pretty sure that my method is correct and sometimes they recommend ZCA whitening.
Another problem is that there are too many heuristic issues to learn meaningful features with CRBM.
I'm expecting a set of well-oriented gabor filter which H. Lee et al. have shown in their paper.
Authors used sparse coding and grouping nearby hidden units to achieve a set of well-oriented gabor filters from the natural images.
I'll write down some ambiguous parts of my implementation
Block-process by grouping nearby hidden units with pooling ratio. I used softmax to infer p(h|v) using the equation in the ICML2009 paper, and sampled hidden state like one normaly does in RBM. double(p(h|v) > rand( sizeof ( h ) ) ). After the pretraining step of the 1st layer CRBM, I will freeze my parameters and use them to get hidden layer activation and then do max-pooling to feed the pooled layer to 2nd layer input. I'm not quiet sure to use the probability itself or sample the states and then shrink the hidden layer into 1/c. (I used sum value of the hidden activations in block 'A' for the activation of corresponding pixel 'a' of pooling layer)
I don't know my parameters are learned properly. I have no idea about typical size of pretrained weights and biases of sparse CRBM. Well, I think it depends on many other things, but at least I need an intuition that I'm doing a right job. I am using weight decay in weight update term. I simply added
(minus)(weight.decay * weight) term to weight gradient.
Deciding when to stop the training. Is their any reasonable idea to choose the stopping point for the training of CRBM?
Is it possible to do backpropagation(fine-tuning) on CDBN? and does it work well? Can somebody explain me about the fine-tuning of CRBM?
Thank you :)
Mar 02 '12 at 12:05