|
I would like a meaningful and concise explanation for what it means exactly, when someone says, "We built a statistical model of all our images". I overheard this, (and keep overhearing that phrase), but I am not sure how/what that entails exactly. Some background: What I am aware of are techniques whereupon one will try to fit parameters to data, given that you assume they come from a certain PDF. For example, if I assume my data is gaussian distributed, then I can use maximum likelihood to estimate its mean and variance. I am also aware of more sophisticated techniques such as EM, where we estimate parameters of a mixture of gaussians for example. (Or any other PDF we want). (I am also aware of PCA, and empirical basis recovery, etc). So I have all the above background, but what I do not get is what it means when someone says we "built a statistical model" for something. Does that mean they have come up with a new PDF? Or is that just another way of saying they have found the underlying parameters of some assumed PDF? Would a fair question to be posed to someone who says this be, what PDF are you assuming, in this case? Thank you. |
|
A statistical model in the most basic sense is something which assigns valid probabilities to things. So when someone says "I built a statistical model of X" they mean that they built a function which, given any element in X, assigns it a probability, such that all probabilities are positive and sum to one. Hopefully, when one builds a model of some data set, the probabilities of things (both items and features) that were actually observed is bigger than the probability of things which were not observed, and the probabilities of similar things are similar. Given a statistical model there are many things you can do with it. A basic one is sample from it, and you can tell how good a model of, say, faces, is by looking at whether samples from the model actually look like faces. Another way of telling how good a model is is by holding out some data points and looking at the model's probability of those data points, the higher it is the better the model, as a good model will assign a high probability to things which behave like real data. Most interesting models, however, have some structure in the probability distribution function, so inspecting that structure can bring more insights about the data. The model might capture the general axes of variation, group words into semantically coherent topics, etc, as a byproduct of assigning high probabilities, and this latent structure is often useful. Finally, if you data is complex, having a model of it allows you to fix some random variables and predict values for the others, which is the main goal of machine learning. |
