|
Dear Group, I was reading through Maximum Likelihood Estimation(MLE) and Expectation Maximimization(EM) algorithm and encountered two Tutorials. One is by Detlef Prescher Titled "A Short Tutorial on the Expectation-Maximization Algorithm" and another was a note from Lingpipe. I want to fit them with Naive Bayes and HMM but now which version should I follow? Are they same with different notations or just different? If any one of the learned members of the room can kindly answer me the question. Best Regards, Subhabrata Banerjee. Gurgaon, India.
This question is marked "community wiki".
|
|
Try looking it from the problem of optimizing the parameters of your distribution.
There are different ways to do the inference, EM is just one of them.
This answer is marked "community wiki".
|
|
Dear Sir, OK. You might have got confused with my question. EM is for iteration till you get a fitting result. EM fine tunes MLE. This I believe understood fine? My question was, for this set of MLE and EM I am getting two tutorials. One is the Tutorial by Prescher and the other one is Lingpipe tutorial. Presecher I practiced all the examples, look nice and my friends locally are suggesting me to follow lingpipe. I am a Linguist and trying to learn them. I want to fit MLE and reciprocal EM in Naive Bayes to build a PoS Tagger and HMM is also for the same purpose. I would then try to compare the results of the Naive Bayes and HMM. If you want to know anything more please let me know. I would be happy to answer. Best Regards, Subhabrata Banerjee.
This answer is marked "community wiki".
Indeed, I got confused by your question... Nope, EM does not fine tune MLE. I would recommend you to take a look at text books such as Bishop's PRML, to clear these things out. You should take a look at what a likelihood function is, how one can "learn" or infer parameters (MLE, MAP, fully Bayesian, etc) of a distribution. Take a look also at latent variable models, like HMM (most books on machine learning have a section on HMMs), and what the problems are with this kind of models. If you are intending to implement a POS tagger, you won't actually necessarily need EM. You will have a tagged corpus, and you can obtain the empirical distributions by simply counting words and POS tags. Further, you might also try existing implementations of POS taggers based on HMM and NB (afaik NLTK offers both implementations), before implementing your own version. But if you are really keen in learning more about machine learning POS taggers, you will find it very entertaining to read about the aforementioned topics.
(Mar 27 '11 at 15:23)
Breno
|
|
Dear Sir, Thank you for your kind answer. I would try to take a look at the book you kindly suggested. But if you can suggest some good online material with some examples or exercises that would be very nice. Yes, I have built annotated corpus. Simple Tagger building I know. But I want to build the ML approach. I am trying NLTK and as I code in Python so it would be fine. Meanwhile, if you can find any good online book with some exercises and lessons please let me know. Wishing You A Happy Day Ahead, Best Regards, Subhabrata.
This answer is marked "community wiki".
Well, wikipedia is also a good reference. For more elementary discussions on probability theory and inference you could also check MacKay's book on information theory and machine learning (http://www.inference.phy.cam.ac.uk/mackay/itila/). Or you could take a look into David Barber's book on Bayesian reasoning and machine learning, still in draft form, available at http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Main.Textbook. Here is a reference for HMMs: http://metaoptimize.com/qa/questions/5245/what-are-some-good-resources-for-learning-about-hidden-markov-models
(Mar 27 '11 at 15:46)
Breno
|
|
No, they are not the same thing. MLE and EM are not comparable, actually. You can use EM to obtain a ML estimate, for example. Could you please be more specific in you question? Are you trying to implement a NB classifier? What are you planning to model with an HMM?
This answer is marked "community wiki".
|
|
So nice of you. If my report clears the desk, as I may not promise advertisement, but I would surely consider donating some amount so that many researchers may find it helpful including me for my future questions. I am a member of Python community so some friends from there kindly helped me with the name of your room. Wishing You A Great Day Ahead, Best Regards, Subhabrata Banerjee.
This answer is marked "community wiki".
|
|
Thanks. One of your nice colleague in the room had just pointed my faults and kindly suggested me some tutorials. I am reading them and solved the issue. Thank You for kindly permitting your valuable time to solve my issue. Wishing You A Nice Day Ahead, Best Regards, Subhabrata Banerjee.
This answer is marked "community wiki".
|
|
Dear Sir, Thank you for your kind and prompt reply. I would go through them. I have one small request I am researcher with an Indian Institute named Indian Institute of Science, Bangalore. I may be planing soon one technical report on the topic. So, if you can kindly send your name and designation, I would be delighted to acknowledge you in the report. Wishing You A Nice Day Ahead, Best Regards, Subhabrata Banerjee.
This answer is marked "community wiki".
I don't feel comfortable being acknowledged for such little help. But feel free to acknowledge and advertise metaoptimize.com/qa.
(Mar 27 '11 at 15:59)
Breno
|
