|
Hi, I admit I am a newbie to programming. But I think this question is quite practical. I know if I want to prevent the denominator being zero, just add 1 to every denominator in the whole sequence. But when I calculate the log of a number which is in the range of (0,1), naturally the result is negative. How can I prevent this happening? If I add 1 to this number in the whole sequence, does this make sense? And my further question is whether there is a 'secret' manual to talk about this stuff, I know this is totally experience-talking. I am just wondering if these kind of manual exists. |
|
Adding one to the numerators before normalizing is refered to as smoothing. I don't understand what you mean by avoiding the fact that the log of a number between 0 and 1 is negative. This is not usually a problem. If you want to know how to normalize log-probabilities, Alex Smola has a blog post that explains what is usually done. Does this answer your question? Thank you, this is what I want.
(Oct 03 '10 at 19:41)
Zhibo Xiao
|
log-probabilities are always 0 or negative. That's why formulas like log loss or entropy use -log(p) instead of log(p)