Recommended reading on model selection and model evaluation?

There's a previous question on this site focused on generative bayesian model selection. If you want something more general than that, you could look up information on AIC, BIC, etc. If all you have is a classifier, then you probably should select your model based on some sort of generalization bound. John Langford has a tutorial that summarises most practical results in this area. The general take from that is that if you do have a test set that is independent from your training set, your best bet is to use it to do model selection, but if you don't, there are some less accurate models as well.

A nice bibliography consisting of most of the popular model selection approaches is here: http://www.modelselection.org/

Here are some papers on model selection I personally found interesting