I'm starting this question to collect a list of good freely available textbooks that concentrate on some aspect of machine learning (rather than applications of ML -- these can go in separate questions). Good tutorials could be included as well, but I'd rather not have, say, slides that are a bit cryptic on their own.

  • The Elements of Statistical Learning is a great text covering most core topics in supervised learning along with a bit of unsupervised learning and some other specialist areas (like high dimensional problems).
  • Information Theory, Inference, and Learning Algorithms covers what the title suggests. It doesn't have a great deal of depth on the machine learning topics but is good for an overview of the techniques used in Bayesian inference.
  • Gaussian Processes for Machine Learning is the definitive reference for Gaussian processes.
  • Data-Intensive Text Processing with MapReduce contains patterns for implementing your algorithm in the Map-Reduce framework.
  • Reinforcement Learning: An Introduction covers the fundamentals of bandit algorithms and reinforcement learning in fully observable worlds (MDPs). Note it says very little about generalisation and practically nothing about acting in partially observable worlds (POMDPs). Since this book was published there has been substantial work in all areas of reinforcement learning; while the book will give you the basics you'll have to do a lot of reading in the literature to catch up to current work.
This question is marked "community wiki".

asked Jul 01 '10 at 07:19

Noel%20Welsh's gravatar image

Noel Welsh

edited Nov 11 '12 at 17:00

Thanks for this. Will be a great resource.

(Oct 19 '12 at 05:04) RecurseMeta

The correct title for the first reference is "The Elements of Statistical Learning"

(Oct 20 '12 at 13:21) Gabriel Kronberger

10 Answers:

D.Barber: Bayesian Reasoning and Machine Learning


This answer is marked "community wiki".

answered Jul 01 '10 at 09:24

Laszlo%20Kozma's gravatar image

Laszlo Kozma


I started reading this text after you recommended it and I have to admit, it is very intuitive. A plethora of excellent examples are contained, and the accompaniment of both code and visual explanations is incredibly valuable.

(Jul 03 '10 at 20:59) Daniel Duckwoth

Just Thought I'd remind everyone of Andrew Ng's CS229 Machine Learning course notes. Comes with video lectures.

This answer is marked "community wiki".

answered Jul 08 '10 at 13:17

Ron%20Shigeta's gravatar image

Ron Shigeta

edited Oct 16 '10 at 12:12

Noel%20Welsh's gravatar image

Noel Welsh

Amazing lecture notes, covers a very broad range of machine learning topics.

(Jun 05 '11 at 20:21) Alex Measure

answered Jul 01 '10 at 07:27

Ravi%20Mohan's gravatar image

Ravi Mohan

edited Jul 01 '10 at 07:58

I didn't know RL was available. Since this is a book I have read I've added it my list above with some commentary. If you have comments on the other books it would be great to see them. (This later part applies to all responses.)

(Jul 02 '10 at 05:40) Noel Welsh

Introduction to Machine Learning course notes (based on a forthcoming book by Shai Ben-David and Shai Shalev-Shwartz) - 110 pages of comprehensive notes. The notes have a slight bias towards the more theoretical side of ML - PAC, Rademacher complexity, etc., with theorems and all that.

This answer is marked "community wiki".

answered Jul 08 '10 at 07:09

ualex's gravatar image


Wow, these notes seem really awesome! They cover many topics, but in a very concise manner!

(Jul 12 '10 at 09:32) Hugo Penedones

Mike Jordan and Martin Wainwright's Graphical Models, Exponential Families, and Variational Inference


This answer is marked "community wiki".

answered Jul 03 '10 at 04:35

Georgi's gravatar image


I read the paper a while ago and although it's definitely not a light read, it is a must-read for anyone interested in statistical machine learning.

(Jun 24 '11 at 23:37) aristotle137

Also worth to take a look at these very neat handouts from

and the ML book by Tom M. Mitchell.

This answer is marked "community wiki".

answered Aug 24 '10 at 14:40

chl's gravatar image


I want to add another one to the list.

It is still a draft, but is written by Alex Smola, which means quality.

Introduction to Machine Learning

This answer is marked "community wiki".

answered Jun 22 '11 at 18:52

Leon%20Palafox's gravatar image

Leon Palafox ♦

Excellent reference, thank you.

(Jun 23 '11 at 10:30) Lucian Sasu

Probabilistic Models in the Study of Language by Roger Levy at UCSD. Still in draft form so incomplete (some chapters have yet to be written), but freely available at http://idiom.ucsd.edu/~rlevy/textbook/text.html.

This answer is marked "community wiki".

answered Jul 05 '10 at 13:22

Jason%20Adams's gravatar image

Jason Adams

edited Jul 05 '10 at 13:22

The link is broken; the correct link is http://idiom.ucsd.edu/~rlevy/pmsl_textbook/text.html

(Oct 19 '12 at 11:48) Adnan Masood

a very good draft of the book to be released: A first course in Machine Learning + Excellent lecture notes.

This answer is marked "community wiki".

answered Jul 08 '10 at 14:33

Julio%20Cesar's gravatar image

Julio Cesar

edited Jul 08 '10 at 14:34

The link you gave is not working (Error 404). Could you correct it, please?

(Jan 30 '11 at 06:27) Lucian Sasu

I think this book has been published and thus pulled out of web.

(Aug 17 '11 at 17:32) Anand

i always recommend this list, not all of these books are freely available but you can always use your local university library if you're short on cash: http://measuringmeasures.com/blog/2010/3/12/learning-about-machine-learning-2nd-ed.html

This answer is marked "community wiki".

answered Jul 08 '10 at 14:06

alex's gravatar image


edited Jul 08 '10 at 14:11

Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.