Good Freely Available Textbooks on Machine Learning

I'm starting this question to collect a list of good freely available textbooks that concentrate on some aspect of machine learning (rather than applications of ML -- these can go in separate questions). Good tutorials could be included as well, but I'd rather not have, say, slides that are a bit cryptic on their own.

The Elements of Statistical Machine Learning is a great text covering most core topics in supervised learning along with a bit of unsupervised learning and some other specialist areas (like high dimensional problems).
Information Theory, Inference, and Learning Algorithms covers what the title suggests. It doesn't have a great deal of depth on the machine learning topics but is good for an overview of the techniques used in Bayesian inference.
Gaussian Processes for Machine Learning is the definitive reference for Gaussian processes.
Data-Intensive Text Processing with MapReduce contains patterns for implementing your algorithm in the Map-Reduce framework.
Reinforcement Learning: An Introduction covers the fundamentals of bandit algorithms and reinforcement learning in fully observable worlds (MDPs). Note it says very little about generalisation and practically nothing about acting in partially observable worlds (POMDPs). Since this book was published there has been substantial work in all areas of reinforcement learning; while the book will give you the basics you'll have to do a lot of reading in the literature to catch up to current work.

This question is marked "community wiki".

asked Jul 01 '10 at 07:19

Noel Welsh
641●3●8●20

edited Jul 02 '10 at 05:48

10 Answers:

oldest newest most voted

12next page »

D.Barber: Bayesian Reasoning and Machine Learning

http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Main.Textbook

This answer is marked "community wiki".

answered Jul 01 '10 at 09:24

Laszlo Kozma
1●3●5●8

I started reading this text after you recommended it and I have to admit, it is very intuitive. A plethora of excellent examples are contained, and the accompaniment of both code and visual explanations is incredibly valuable.

(Jul 03 '10 at 20:59) Daniel Duckwoth

Just Thought I'd remind everyone of Andrew Ng's CS229 Machine Learning course notes. Comes with video lectures.

This answer is marked "community wiki".

answered Jul 08 '10 at 13:17

Ron Shigeta
1●1●1●2

edited Oct 16 '10 at 12:12

Noel Welsh
641●3●8●20

Amazing lecture notes, covers a very broad range of machine learning topics.

(Jun 05 '11 at 20:21) Alex Measure

Convex Optimization – Boyd and Vandenberghe

Reinforcement Learning - An Introduction

Introduction to Information Retrieval

answered Jul 01 '10 at 07:27

Ravi Mohan
136●1●1●5

edited Jul 01 '10 at 07:58

I didn't know RL was available. Since this is a book I have read I've added it my list above with some commentary. If you have comments on the other books it would be great to see them. (This later part applies to all responses.)

(Jul 02 '10 at 05:40) Noel Welsh

Introduction to Machine Learning course notes (based on a forthcoming book by Shai Ben-David and Shai Shalev-Shwartz) - 110 pages of comprehensive notes. The notes have a slight bias towards the more theoretical side of ML - PAC, Rademacher complexity, etc., with theorems and all that.

This answer is marked "community wiki".

answered Jul 08 '10 at 07:09

ualex
131●2●5●9

Wow, these notes seem really awesome! They cover many topics, but in a very concise manner!

(Jul 12 '10 at 09:32) Hugo Penedones

Mike Jordan and Martin Wainwright's Graphical Models, Exponential Families, and Variational Inference

http://www.nowpublishers.com/product.aspx?product=MAL&doi=2200000001

This answer is marked "community wiki".

answered Jul 03 '10 at 04:35

Georgi
1●1●2

I read the paper a while ago and although it's definitely not a light read, it is a must-read for anyone interested in statistical machine learning.

(Jun 24 '11 at 23:37) aristotle137

I want to add another one to the list.

It is still a draft, but is written by Alex Smola, which means quality.

Introduction to Machine Learning

This answer is marked "community wiki".

answered Jun 22 '11 at 18:52

Leon Palafox
3146●54●71●108

Excellent reference, thank you.

(Jun 23 '11 at 10:30) Lucian Sasu

Also worth to take a look at these very neat handouts from

David Banks about Data Mining
Maneesh Sahani about Probabilistic & Unsupervised Learning
Robert Nowak about Statistical Learning Theory

and the ML book by Tom M. Mitchell.

This answer is marked "community wiki".

answered Aug 24 '10 at 14:40

chl
91●2●8

Probabilistic Models in the Study of Language by Roger Levy at UCSD. Still in draft form so incomplete (some chapters have yet to be written), but freely available at http://idiom.ucsd.edu/~rlevy/textbook/text.html.

This answer is marked "community wiki".

answered Jul 05 '10 at 13:22

Jason Adams
1●4

edited Jul 05 '10 at 13:22

a very good draft of the book to be released: A first course in Machine Learning + Excellent lecture notes.

This answer is marked "community wiki".

answered Jul 08 '10 at 14:33

Julio Cesar
1●1●3

edited Jul 08 '10 at 14:34

The link you gave is not working (Error 404). Could you correct it, please?

(Jan 30 '11 at 06:27) Lucian Sasu

I think this book has been published and thus pulled out of web.

(Aug 17 '11 at 17:32) Anand

Introduction to Machine Learning: Draft of Incomplete Notes by Nils J. Nilsson

Although I have tried to eliminate errors, some undoubtedly remain---caveat lector. Certain elements of the typography (overflow into margins, etc.) have not been polished.

The notes survey many of the important topics in machine learning circa the late 1990s. My intention was to pursue a middle ground between theory and practice. The notes concentrate on the important ideas in machine learning---it is neither a handbook of practice nor a compendium of theoretical proofs. My goal was to give the reader sufficient preparation to make the extensive literature on machine learning accessible. The draft is just over 200 pages (including front matter).

Though, (as Nilsson notes) it is a bit dated, the text has some good examples and explanations for those new to ML.

This answer is marked "community wiki".

answered Jul 01 '10 at 19:35

John L Taylor
615●4●15●18

edited Jul 01 '10 at 20:01

12next page »

Your answer

toggle preview

community wiki

Subscription:

Once you sign in you will be able to subscribe for any updates here

Tags:

machine-learning ×184
books ×12
textbooks ×6
free ×2

Asked: Jul 01 '10 at 07:19

Seen: 55,022 times

Last updated: Aug 17 '11 at 17:32

Good Freely Available Textbooks on Machine Learning

Subscription:

Related questions