Until now I was working on Binary-Binary RBM and now I want to extend my implementation to support Gaussian-Binary RBM for real-valued data.

I have normalized my input data to zero mean and unit variance.

Here is my CD1 step:

activate_hidden(h1, v1)
activate_visible(bernoulli(h1), v2)
activate_hidden(h2, v2)

Before, the values of visible units (pi) were reconstructed using sigmoid(x), x being its total input plus its bias. Is it correct that now I can simply use x for pi ?

Moreover, when I sample the hidden values given the visible values, do I directly sample using the real-valued values or is there some processing here ?

And last question, do I still have to use Bernoulli of the hidden values when activating the visible units from the hidden units ? I saw some implementation were adding noise here, but I understood that since I have zero-mean and unit variance input data, I could avoid adding noise.

Thanks

asked Jun 05 '14 at 04:59

Baptiste%20Wicht's gravatar image

Baptiste Wicht
31121315

edited Jun 05 '14 at 05:01


One Answer:

You cannot reconstruct using sigmoid since the values will be restricted to [0,1]. So you use x.

answered Jun 05 '14 at 06:08

Ng0323's gravatar image

Ng0323
1567915

Ok, Thanks. Do you have any idea about the other two questions ?

(Jun 05 '14 at 07:17) Baptiste Wicht
Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.