|
I have read my share of machine learning literature. Its not a lot, probably very little as compared to the vast amounts that is out there and what people here at Meta have read. But I have never seen a book that describes machine learning by promoting strong intuition at first and then pushing in the theory later. Theory is very important no doubt but motivation and intuition is sometimes equally important. Metaoptimize is awesome, I have seen explanations on this web site that strongly promote intuition on things like Why Projecting data into Hilbert space is useful, Proxy losses, The guts of how Spectral and Eigen based decomposition methods work... But here is what IMHO I feel we should do to make Machine learning easier for the masses...Its important to get more people involved, but more important to teach it right the first time... Machine Learning needs at least some mathematical base. So its tough to have "Machine learning for dummies" kind of teaching methodology, one needs to know some linear algebra and calculus... Nevertheless most people who understand the basics find it hard. It is hard but the motivation to learn it should not be! So considering we did well at basic linear algebra and calculus, there should be a "Head first"(awesome reads BTW) way of motivating these with nice snappy python/matalab programs and drawings explaining the crux of things like what is proxy loss and why is one better. And this simple visual cue of why eigen decomposition method of clustering work the way they do. If some had told me in ML 101 first that what Eigen decomposition does is give me an orthogonal basis and then cluster the data points like in the bottom right of that figure, and then the mathematics about eigen analysis, I would have been motivated much more... Instead I had to hunt around and scavange for intuition... This kernel tric answer(Alexandre's answer) is another simple to understand example of why SVM's and so many other machines can make use of the kernel... Another very important thing is mentioning the things that matter and simple optimization trics and why they would work... A classical example would be how deep learning algorithms can work well for neural networks as well and how can be improved by using the Gibbs sampling plus the conjugate prior trick. Now the theory is quite complicated; 3-4 weeks to read on conjugate priors and RBF's atleast. But just the algorithm implementation is not complicated and its visualization is very motivating. If someone tells you just make this adjustment to your neural net to make it better you would do it in a snap (Geoffs way of explaining is awesome BTW in those video talks of his. One should read the paper and listen to the video to understand the Deep learning breakthrough). Of course this all can only go so far... The basics of ML cant be done without learning substantially more... Things like deep belief nets, SVM's etc have substantial theory to really understand them... But is it not easier mentally to draw intuition before digging in deep? I strongly feel that there is enough material out there to teach the basics of machine learning in a motivating way. We must make things simpler... Like here on Meta.. Time for a "head first Machine learning" book? I feel some one should gather this all up into one book... I dunno.. Tell me what you think... I do not mean to intend in any way to disregard the importance of sound theory here and my apologies for any such vibe my post here gives, theory is invaluable. I just wanted to put my thoughts here...
showing 5 of 6
show all
|
There somewhat of a head first machine learning book, it's Programming Collective Intelligence, by Toby Segaran. It is, however, a lot more superficial than what you hint at in this question.
Something like what I want would probably take more space to describe. The basics are not that hard, its the inner workings and the intuition as to "why it works like it does" , that is missing....
I agree with you. I think that maybe the central difficulty in making machine learning globally accessible is that it fundamentally requires a form of thinking that is counterintuitive for most people and, specially, relies on precisely the sort of mathematical background that many computer-sciency-type people feel the need to do without. There is a disconnect between most people's intuition and state of the art techniques, and if I've managed to steer my intuition in the right direction it was through hitting my head on the wall enough to build other sorts of foundations, and still a lot of the state of the art flies way over my head.
I think what you describe is hard to do precisely because one must fight the temptation to make things seem simpler by glossing over the details.
I would think that this is more of a general problem when it comes to teaching and learning maths. In college when I had linear algebra classes, they were completely void of intuition, but just learning procedures for (multiplying, the decompositions), that's why most CS-guys are left with the impression that they just need to do it now and never again. So they'll never know about the full potential of having a good linear algebra background and being able to use it. Fortunately I found the linear algebra video lectures from Gilbert Strang, which fixed my background. So there's no way around it, but to first make maths more than just a bunch of formulas and procedures completely disconnected from the real world.
I personally don't like these books which oversimplify things like (Collective intelligence or algorithms for the intelligent web), and not a good resource for studying ML perse. But maybe some find them useful, don't know. Metaoptimize is filling a big gap, and it's a really precious resource for more of a real-time collaboration.
MetaOptimize is awesome. People like Alexandre are doing a great job. I feel that if you a pick a good book (PRML - Bishop, ML - Mitchell) and really give it some time, the intuition will come to you. Most of the good books do explain the motivation behind the models before introducing them. It's a matter of patience. Also, I think a better way of getting intuition is to compare different models, instead of just reading about each one in isolation. Finally, ML is a high level subject. You can teach English to a person, but you cannot make a Shakespeare from a book.
Yes it is a matter of patience and a continuous cycle between going through the books and to the computer model and your pencil and paper. And i believe we truly have those moments of understanding when we understand the math and the intuition together. Point is that that intuition is seen mostly through the complex verbiage and mathematics. People at Meta have already given examples that are not so complex in verbiage or mathematics.
I dont believe these intuitions are that few that they are not worth mentioning in books... I know the mathematics and the research is hard and must be given the due, but it is very motivating to have good intuition.
And, again, i am not talking about intuition on just simple things like maximum margins or Examples in conditional independence using typical earthquake and alarm examples... I have observed questions on Meta and I feel that there is a genuinely good intuition about very complicated things...
Yet very few books mention those and if they do they rarely do it as consistently. This intuition must be there and must be followed by the share of complex mathematics it deserves and it will fill lots of pages... There is no point or usefulness to be had from dumbing it down to "Machine Learning for Dummies" or in 24 hours...
Bishop, Duda & hart are excellent books, every time i go to them after a few weeks, I understand something new... Somehow its generally after being on Meta or hunch.net, the blogs or the IRC channels...