|
I am having a multi dimensional dataset and I am trying to solve a classification problem. Some features are discrete valued. How can I apply logistic regression on these discrete valued features? or can I apply it? Wikipedia mentions that it can be applied. Also, are there other algorithms for classifying based on discrete values, I should be considering? PS: I am new to data mining. You can simply point me to some readings/ Thank you. |
|
For input features with values drawn from a discrete set of reasonable cardinality N, you can encode them as binary vectors with N dimensions. To represent the value i, set the ith dimension of the vector to 1 and all other dimensions to zero. I call this a "softmax" encoding or "one-of-N" encoding or "one-hot" encoding. If the inputs are from a larger discrete set, you may want to do the same thing conceptually, but replace the multiplication by the weights with a table lookup. If you need substantial parameter sharing for statistical reasons/data sparsity reasons, you will need to change the model or go down the Bayesian path. i wouldn't call this softmax which is usually written as frac{e^{x_i}}{sum_j {e^{x_j}}}
(Nov 11 '11 at 22:11)
Travis Wolfe
Travis, this is called softmax because it's how you encode a variable you want to predict with a softmax function (that is, if your classifier is a matrix W and P(y|x) = softmax(Wx), y is encoded as above).
(Nov 12 '11 at 18:52)
Alexandre Passos ♦
|
|
if the data is discrete but still ordinal, you may want to try adding it straight in, or apply a positive function to it (like rank, x^2, x^0.5, e^x, log(x), etc). this can outperform one-of-k encoding if your assumptions are reasonable, especially if k>2. The data is not ordinal for most part of it. Any ideas on categorical data?
(Nov 12 '11 at 02:59)
zubin mehta
just to be clear, categorical data can be ordinal. you may be using 'categorical' to mean non-ordinal. if you data is truly non-ordinal, then you pretty much have to go with 1-of-k.
(Nov 12 '11 at 10:13)
Travis Wolfe
|
