|
Suppose we have a multi-class classifier that emits a confidence score as well as a class prediction. When evaluating it on the test set, suppose we want to throw out examples for which the confidence is less than some threshold t. Is there a technical term for the percentage of examples that remain? I'm calling this "coverage" right now. There are some applications where what you care about is the "coverage" you can get at a specific accuracy (e.g., suppose you report coverage at 99% accuracy), and I'd like to know if there's an existing word for this concept so that I can Google it and / or use the right term in papers. |
|
"Coverage" is indeed the term I've often seen, so people show a rejection-accuracy curve and say things like "90% accuracy at 80% coverage, and 99% accuracy at 50% coverage". Do you know of any papers that use the term in this way? In the context of recommender systems, coverage seems to refer to the % of topics that a system is able to provide answers for. In the context of machine translation it seems to be the number of sentences in a parallel corpus that are correctly aligned.
(Jul 11 '13 at 15:47)
Ian Goodfellow
and in "Data Mining for Scientific and Engineering Applications" by Robert Grossman, "coverage" seems to be a synonym for "recall" in a detection system.
(Jul 11 '13 at 15:50)
Ian Goodfellow
OK, I found a source that uses "coverage" in this way: http://robotics.stanford.edu/~ronnyk/glossary.html Thanks!
(Jul 11 '13 at 15:51)
Ian Goodfellow
|
