|
Hello, I am attempting to implement contrastive divergence on an RBM with gauss visible units. I have looks around the internet to find out what the updates should be but haven't found a conclusive answer. Below is my guess as to how to compute the updates. It would be greatly helpful if I could get feed back on if it's correct or not. Thanks! So for w_ij the update in CDk is Expectation(vi_0 * hj_0) - Expectation(vi_k * hj_k) In the binary case vi_0 is sigmoid(h * W) My guess is that in the gauss case (in which the data was preprocessed to have zero mean and unit variance) vi_0 is just h*W Is this correct? Thanks! |
|
Short answer: yes Long answer: Depends on how you parameterize the energy function and if you are trying to learn the variances. If the variances of the visibles are not pinned to a value of 1, then they appear in the conditional distributions of v given h and vice versa. I say it depends on the parameterization of the energy function because there are at least two ways to parameterize a Gaussian-Bernoulli RBM, see: Cho, Ilin, Raido, 2011, "Improved Learning of Gaussian-Bernoulli Restricted Boltzmann Machines", and they result in different equations for the conditional distributions. But, again, if you assume the normal distribution of the visible units has variance 1, then YES. |
