|
As I studied on the papers, when we are training DBNs using RBMs, every connection weight matrix (W) should be initialized by its bellow connection weight matrix. This way, it will be guaranteed that adding more hidden layers will improve the final generative model. However, in a Master Thesis it was proven that W2 should be initialized by W1 while in a presentation it was shown that W2 should be initialized by the transposed W1. So, my questions are that 1- which one is correct? 2- In both cases, the dimensions of W1 and W2 are not matched with those are reported in papers. In the first case, the number of hidden units in all hidden layers should be the same as the number of visible units. And, in the second case, the number of hidden units in the first hidden layer can be chosen arbitrary while the number of hidden units in the second hidden layer should be the same as the number of visible units. Thank you in advance, |
|
When you create a 4-5-6-7 network you should have 3 rbm layer and the weight matrix is 4x5 5x6 6x7. Although I'm still struggling with some issue of DBN you can check my code https://github.com/mistree/GoDeep/ Generally it should be correct. |
