How to study Machine Learning

I read lots of papers. In the university, my stack of papers for a week was usually a lot bigger than the tiny stacks of papers my colleagues were annoyed that their advisor had them read before graduating. Also, I tend to do a lot of thought experiments trying to understand how things work and why, and why they are not just done differently. A part of this is spending some time explaining things to myself so I can understand them better (and in practicing this metaoptimize has been extremely helpful, as here I can also get feedback for my attempts at explaining things to understand them). I find helpful to think of alternatives when I read a paper or watch a talk, and try to convince myself about the quality of my alternative suggestions.

I code things far less often than I should, specially because I don't have many datasets lying around to try stuff on, or many ways of measuring the performance of what is happening. When I do run experiments I prefer to code almost everything from scratch, as then I can understand what works, when, and why, and also understand all the little points about optimization that are mixed in the papers with the actual model structure.

When I stated doing machine learning about two years ago, I started implementing things on CUDA as my advisor wanted me to. That might have not been the best start. Since then I tried to understand many different methods from kmeans to hierarchical non-parametrics. Often I implement algorithms and play with them to gain better understanding. Now I mostly use python for that, sometimes with weave. I often browse papers from the big conferences to get a feel of what's going on and alternate between getting to know new ideas and really understand the basics. I barely use other peoples implementations. The libraries I use the most are libsvm and shogun, since I don't want to do SVM optimization - and some computer vision features. I feel my way got me a pretty good and wide insight into ML methods but I didn't produce much myself :-/

This has not been mentioned quite a lot here, but there are some good exercises/questions in some classic machine learning text books. Doing the exercises or the derivations, though time consuming, is helpful in understanding different aspects (e.g. computational complexity) of a model.

Beyond learning machine learning algorithms and models itself, it is always interesting to study problems in other domains, such as biology, vision, neuron science. These fields contain many problems where computational modeling should have been helpful but there is a big gap between applying standard ML algorithms and solving the domain problems.

Most things I wanted to say have been said already but I wanted to emphasize what I think is an important attitude towards all the information we are flooding ourselves with.

I think it is very important to keep a certain form of humbleness at all times. With this I mean that it is important to continuously question your own understanding of the things you read about. There is no 'teacher' to correct us when we misunderstood something. For this reason it also doesn't hurt to still read sections in papers that seem to deal with things you already know. I'm often enough guilty of skipping those sections but I found that they sometimes fill in gaps in my understanding that I didn't know about before or even fix things I misunderstood. I learned that there is a lot more to know about 'simple' methods and models and the basic probability distributions than there is in the standard machine learning textbooks.

Metaoptimize is very good for this because it also forces you to ask yourself whether you can actually be 100% sure when you answer a question. If you cannot be 100% sure it is a nice motivation to (re-) read about the subject matter.

I've got a consulting gig just in order to learn some optimization methods in depth. The best way to learn, in my opinion, is to start with a problem, a nasty real-world problem with its messy data and tight implementation schedule/budget and try to solve it.

Talk to people outside of your usual environment, they may point you to great real-world problems to solve that you'd otherwise miss, buried in a stack of papers. Talk to people with problems, not people with solutions.

The paper that got me into Machine Learning was the Yann LeCun's convolutional networks. I was in my third semester undergrad, and you could imagine back then it blew my mind. So I read pretty much anything I could find on neural networks, because you could actually model the human brain with those. Then I took every course on machine learning I came across. As for implementation, there are plenty of open source implementations. I would read a paper, try to find an implementation of that algorithm and play with it. If it's in matlab, then I would translate it to python and in the process understand what's going on. Once I start feel comfortable with what's going on, I start breaking it down and tweek the parameters. It's just playing and having fun for the most part. Which brings me to answering your question about how to get people excited who want to learn machine learning. I have the feeling there's a sufficient buzz created around this field (no wonder we are in the web 3.0), so people are interested in it for the wrong reasons. So a good test would be to give them some interesting papers to read, and if they come asking for more, and enjoy discussing the problems, then good. But if they come asking for more chewing, ask for simplification or more machine learning for dummies, then there's little hope.