Good Freely Available Textbooks on Machine Learning

I'm starting this question to collect a list of good freely available textbooks that concentrate on some aspect of machine learning (rather than applications of ML -- these can go in separate questions). Good tutorials could be included as well, but I'd rather not have, say, slides that are a bit cryptic on their own.

The Elements of Statistical Learning is a great text covering most core topics in supervised learning along with a bit of unsupervised learning and some other specialist areas (like high dimensional problems).
Information Theory, Inference, and Learning Algorithms covers what the title suggests. It doesn't have a great deal of depth on the machine learning topics but is good for an overview of the techniques used in Bayesian inference.
Gaussian Processes for Machine Learning is the definitive reference for Gaussian processes.
Data-Intensive Text Processing with MapReduce contains patterns for implementing your algorithm in the Map-Reduce framework.
Reinforcement Learning: An Introduction covers the fundamentals of bandit algorithms and reinforcement learning in fully observable worlds (MDPs). Note it says very little about generalisation and practically nothing about acting in partially observable worlds (POMDPs). Since this book was published there has been substantial work in all areas of reinforcement learning; while the book will give you the basics you'll have to do a lot of reading in the literature to catch up to current work.

This question is marked "community wiki".

asked Jul 01 '10 at 07:19

Noel Welsh
726●3●10●23

edited Nov 11 '12 at 17:00

Thanks for this. Will be a great resource.

(Oct 19 '12 at 05:04) RecurseMeta

The correct title for the first reference is "The Elements of Statistical Learning"

(Oct 20 '12 at 13:21) Gabriel Kronberger

10 Answers:

oldest newest most voted

123next page »

D.Barber: Bayesian Reasoning and Machine Learning

http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Main.Textbook

This answer is marked "community wiki".

answered Jul 01 '10 at 09:24

Laszlo Kozma
16●3●5●8

I started reading this text after you recommended it and I have to admit, it is very intuitive. A plethora of excellent examples are contained, and the accompaniment of both code and visual explanations is incredibly valuable.

(Jul 03 '10 at 20:59) Daniel Duckwoth

Just Thought I'd remind everyone of Andrew Ng's CS229 Machine Learning course notes. Comes with video lectures.

This answer is marked "community wiki".

answered Jul 08 '10 at 13:17

Ron Shigeta
1●1●1●2

edited Oct 16 '10 at 12:12

Noel Welsh
726●3●10●23

Amazing lecture notes, covers a very broad range of machine learning topics.

(Jun 05 '11 at 20:21) Alex Measure

Convex Optimization – Boyd and Vandenberghe

Reinforcement Learning - An Introduction

Introduction to Information Retrieval

answered Jul 01 '10 at 07:27

Ravi Mohan
166●1●1●5

edited Jul 01 '10 at 07:58

I didn't know RL was available. Since this is a book I have read I've added it my list above with some commentary. If you have comments on the other books it would be great to see them. (This later part applies to all responses.)

(Jul 02 '10 at 05:40) Noel Welsh

Introduction to Machine Learning course notes (based on a forthcoming book by Shai Ben-David and Shai Shalev-Shwartz) - 110 pages of comprehensive notes. The notes have a slight bias towards the more theoretical side of ML - PAC, Rademacher complexity, etc., with theorems and all that.

This answer is marked "community wiki".

answered Jul 08 '10 at 07:09

ualex
161●4●6●11

Wow, these notes seem really awesome! They cover many topics, but in a very concise manner!

(Jul 12 '10 at 09:32) Hugo Penedones

Mike Jordan and Martin Wainwright's Graphical Models, Exponential Families, and Variational Inference

http://www.nowpublishers.com/product.aspx?product=MAL&doi=2200000001

This answer is marked "community wiki".

answered Jul 03 '10 at 04:35

Georgi
1●1●2

I read the paper a while ago and although it's definitely not a light read, it is a must-read for anyone interested in statistical machine learning.

(Jun 24 '11 at 23:37) aristotle137

Also worth to take a look at these very neat handouts from

David Banks about Data Mining
Maneesh Sahani about Probabilistic & Unsupervised Learning
Robert Nowak about Statistical Learning Theory

and the ML book by Tom M. Mitchell.

This answer is marked "community wiki".

answered Aug 24 '10 at 14:40

chl
91●3●8

I want to add another one to the list.

It is still a draft, but is written by Alex Smola, which means quality.

Introduction to Machine Learning

This answer is marked "community wiki".

answered Jun 22 '11 at 18:52

Leon Palafox ♦
4085●71●94●128

Excellent reference, thank you.

(Jun 23 '11 at 10:30) Lucian Sasu

i always recommend this list, not all of these books are freely available but you can always use your local university library if you're short on cash: http://measuringmeasures.com/blog/2010/3/12/learning-about-machine-learning-2nd-ed.html

This answer is marked "community wiki".

answered Jul 08 '10 at 14:06

alex
408●6●13●18

edited Jul 08 '10 at 14:11

Probabilistic Models in the Study of Language by Roger Levy at UCSD. Still in draft form so incomplete (some chapters have yet to be written), but freely available at http://idiom.ucsd.edu/~rlevy/textbook/text.html.

This answer is marked "community wiki".

answered Jul 05 '10 at 13:22