|
Hi, I am new to Theano. I am trying to do some regression using SGD. I have about 100 features. So my w = random_weights((100,1), noise = 0,01); b = random_weights((1,),noise=0.01) params = [w,b] I am using learn_rate = 0.01 grads = T.grad(cost=cost, wrt=params) updates = [] for p,g in zip(params,grads): updatesappend([p, p - g * learn_rate]) ... Everything works perfect, except that there are lot negative terms and also lot terms with very small number like in the order of 1e-10 I would like to enforce nonnegative on my results, so I change the above to: for p,g in zip(params,grads): updatesappend([p, T.maximum(0, p - g * learn_rate)]) ... It also works pretty well, now there are no more negative numbers in my coefficients. But there are still lot coefficients with number in the order of 1e-10. Ideally, I would like that there are only a few nonnegative coefficients, with the rest to be 0. So I would like to modify the "update" in the above: if the updated value "p - g * learn_rate" is less than 1e-10, then set it to be 0. But that involves to compare p - g * learn_rate to 1e-10. The former is of type theano.tensor.var.TensorVariable, while the later is a float. How do I do that? Thanks for any helps and advices! |
Any solutions?