A question on training of HMM

Consider training an HMM where you have a training set with both state and observation labels. You train the parameters to maximize likelihood, which ends up with a simple procedure for setting parameters from counts, like with Naive Bayes.

Now consider a scenario where your training set does not have observation labels. You can use Forward-Backward to create "artificial training data" and use likelihood maximization as in the scenario above. IE, set your parameters to some initial numbers and get observation labels using forward backward. Those labels will be "soft", and you can treat them as partial counts, ie if forward backward gives 0.5 of state 1 at some position, that will contribute 1/2 to the count of state1 observations. Now you have your training data with labels, and set parameters as in the fully observed case, by maximizing likelihood. If you run forward-backward after these steps, you get different set of "training data", so you repeat the whole procedure until convergence.

A good introduction to Forward-Backward training is Eisner's "An Interactive Spreadsheet for Teaching the Forward-Backward Algorithm"

I can recommend you this lecture notes by a student of Andrew NG

http://www.stanford.edu/class/cs229/section/cs229-hmm.pdf

Here he explains quite well the different algorithms you may use in HMM to get the transition probabilities or the effect probabilities, or even both.

Hope it is of any use

Forward-Backward algorithm is not EM. It is a dynamic programming approach to calculate the probabilities p(x_1,...x_n, z_n) and p(z_n| x_n+1,...) . These probabilities are needed in the M-step of the EM algorithm for HMM. A very readable explanation is in chapter 13 of the "Pattern Recognition and Machine Learning" of Chris Bishop. You can read chapter 9 to understand EM also.