|
As far as I know most multilabel classification methods are for the case when each label is binary (each label takes one of two possible values). Are there any multilabel classification methods that allow each label itself to be multiclass (each label takes one of K possible values)? |
|
Of course there is, there are many generalizations for things like that: You could basically train multiple SVMs for your sets and evaluate the score of the classification (I don't enjoy this approach, but it works), and the best of all is that it can be used for any binary classifier that you might have. (This is called One Vs All) A more elegant approach is to use multiclass Naive Bayes, where you can easily extend for multiple classes by extending the support of the Dirichlet prior you use for the model. One more classic approach is the use of the Sofmax regression, which is the natural extension of the logistic regression, where instead of using the Logit function, you use a sofmax function. Your answer seems to be for the case of single-label multiclass classification. However, in my case I have several labels (possibly correlated) each of which can take >=2 possible values. I was looking for methods that work with such data. Do you know of any?
(Apr 21 '13 at 14:10)
pmqa
When you say multi-label, do you mean multi-task, i.e., each output y_i, i=1,...N (N samples) is a vector?
(Apr 21 '13 at 23:39)
digdug
Multi-label and multi-task seem to be near-synonyms, except multi-label usually means multiple binary classification tasks.
(Apr 22 '13 at 08:00)
larsmans
@digdug: Yes, in my multilabel+multiclass problem, y_i is a vector with each element of the vector taking one of K>=2 values. @larsmans: Yes, but you can generalize multi-label classification problem such that there are multiple multi-class classification tasks. I am looking for any methods that work in such settings.
(Apr 22 '13 at 11:28)
pmqa
If each data point has more than one label, I can only think on hierarchichal clustering.
(Apr 22 '13 at 17:08)
Leon Palafox ♦
|
|
I don't know of any tool that implements this, but in principle you could generalize the standard univariate multinomial regression, i.e., logistic regression with more than two labels (where you only observe one such label at each instance), to multivariate (multi-task) multinomial regression, where there are multiple labels at each instance, each label having one of multiple values. You could also have some group-wise penalization, like group lasso, otherwise this will be the same as K separate multinomial regressions (i.e., no information sharing between tasks). The Elements of Statistical Learning book explains multinomial regression. For multi-task penalization, check out Francis Bach's papers, especially http://www.di.ens.fr/~fbach/bach_jenatton_mairal_obozinski_FOT.pdf. |
You can always use a 1-of-c encoding for multiple labels. I cannot see the difference of multiclass and multilabel?
Can you give an example of this scenario?
Sure, I'm doing something like this for SVHN in fact -- suppose you want to predict 4 digits simultaneously with neural network -- connect 4 output layers to a single shared hidden layer, 10 outputs in each layer