|
What are the key differences between feature extraction, pattern recognition, and dimensionality reduction? Edit: Of these three terms, are there any X is a kind of Y relationships? |
|
Feature extraction: Given a set of objects, perform a function on them that extract features (often numerical though not required), turning those objects into an array of values. As an example, given a text document, count the number of times each word appears from a list. Note that the objects themselves could be feature arrays, and then the feature extraction is about finding better features - this would be a form of dimensionality reduction (as below). Pattern recognition: Given a set of arrays of features (as in the above), work out if there are patterns to the data and whether those patterns can be used to predict (either class values or other feature values). Classification is a type of pattern recognistion, as is clustering, regression and most machine learning tasks. Dimensionality reduction: Reduce the number of features needed to adequately represent each object. From the first example, say our list of words is 50,000 long. Many of these words will be related and storing all values will not be efficient. Running an algorithm such as PCA can reduce the number of features while still comparably representing the objects. +1: Thanks for your answer. I've edited my question with a second question. Also, why do you say that feature extraction on the feature extraction is necessarily a dimensionality reduction? Does it never make sense to extract from a small feature space onto a larger one? (For example, RGB values to a categorical of colour names?)
(Aug 22 '11 at 20:27)
Neil
A lot of what I have read shows that Feature extraction is a type of Dimensionality reduction, but I don't fully agree with that - it is mainly true. Dimensionality reduction would be a type of pattern recognition.
(Aug 22 '11 at 21:52)
Robert Layton
1
And the whole this is artificial intelligence if its not yet working, and data mining if it is.
(Aug 22 '11 at 21:52)
Robert Layton
1
Feature extraction does not have to reduce the dimensionality. Sparse coding for example looks for sparse features, that usually live in a much higher dimensional space than the features you started with. I would say dimensionality reduction is a type of feature extraction - though this doesn't have to be the main goal when doing it. Often it is for computational reasons as Robert said.
(Aug 23 '11 at 01:54)
Andreas Mueller
1
@Robert, @Andreas: Thanks for the answer. I'm still not totally clear on what the difference is between feature extraction and pattern recognition. For example, a Boltzmann machine can be said to do feature extraction, and it can be said to do pattern recognition. Your bag-of-words example is feature extraction, but probably not "pattern recognition." Is there an algorithm that does pattern recognition without doing feature extraction? In other words, is pattern recognition a subset of feature extraction?
(Aug 23 '11 at 15:38)
Neil
You are right. As in the answer, pattern recognition is about what you do with the features once extracted. In machine learning, you will rarely get a high level description that everyone agrees on, and that works in most situations. You seem to intuitively understand the concepts, which means you are pretty much there already. If you have a specific question about how to label a specific algorithm, it doesn't hurt to ask. In general though, the separations are blurry but generally as I've written above.
(Aug 23 '11 at 19:14)
Robert Layton
showing 5 of 7
show all
|
