|
I'm trying to implement HMM algorithms cited in Rabiner tutorial. In doing this, I'm confronting several problems when working if moderate large dataset (10k). The training algorithm is consuming a lot of memory and the parameters tends to zero very fast. It's clear that there is several issues related to numerical stability and sparse data handling. My question is, is there any tutorial or implementation that deals with the problems faced when using HMM for real applications? If there are people interested, I want to implement an open-source python library to serve as a prototyping tool with HMM, including HHMM Also for information extration. I have not found one that addresses HHMM or involves a large volume of data yet. |
|
Hello, Gustavo: I'm in the same situation. I've implemented the Gaussian HMM code. It has serious stability issue. Especially, I noticed that it tends to favor large covariance parameter values. However, after I fixed the covariance parameter values, things work much better. I'm wondering how state-of-the-art is for Gaussian HMM in practice. Do people fix covariance parameter, as I did. Or, do they have fancy updating scheme that mitigate the issue? |
|
That tutorial would not be my first choice to learn how to implement HMMs, sometime ago I followed this tutorial, and got very decent results. |
What is "the training algorithm"? MAP, Baum-Welch?
Baum-Welch
The parameters when doing EM training are normalized after every iteration (as in, you want all your conditional probabilities to sum to one).
You might be experiencing numerical underflow when doing forwards-backwards, so implement that with log probabilities instead of probabilities.
I implemented using the log space and numerical underflow problems were solved. I'm also using smoothing methods for initial probabilities.