I'm implementing the sparsity technique used by Honglak Lee for CRBM. There are two parameters:

  • pbias: The target sparsity
  • pbias_lambda: The learning rate for sparsity

I have some issues for pbias_lambda. On my case (working on MNIST), I have to set it very high (around 1000) in order to reach the target sparsity sometime. When running Honglak Lee's code, the target sparsity is reached very quickly and the pbias_lambda is only 5.

Is that normal behaviour ? How should I go about tuning this ?

Thanks

asked Oct 07 '14 at 05:11

Baptiste%20Wicht's gravatar image

Baptiste Wicht
31121315

edited Oct 07 '14 at 05:12

Be the first one to answer this question!
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.