|
What is the best way to plot ROC/PR curves for a k-Nearest Neighbors algorithm, and e.g. compare them to an SVM? |
|
ROC curve works only for binary classification and only when you have a continuous output from the classifier. The most common case is when the classifier returns a probability value. The meaning of that probability is that when it is above some threshold, the predicted value is considered one of the binary values of the class, when the probability is below the threshold the predicted value is the other class value. You will use as thresholds the unique continues values produced by classifier. In each unique threshold value you compute the ratio between true positive rate and false positive rate, using, of course, the binary predicted values with that threshold and the real binary values. Also, you have to draw lines as a stairs, with steps at each threshold. Without continuous values which can be used to "walk among them" with a variable threshold you simply can't see information ROC can unveil. |
What's the output of your kNN method? Just a class label?
@digdug Yes, just a class label.
To calculate a roc curve you need a continuous output: unless you can get that out of your classifier, you won't be able to proceed. Can you get the k distances out of your classifier as well?