|
What are the current most interesting incomplete or open research topics/questions/ideas in the domain of supervised and semi-supervised learning for classification tasks (and more precisely in online learning settings), that one could address ? |
|
Direct empirical cost minimization (that is minimization of empirical risk with non-continuous cost functions directly without the use of surrogate loss functions) seems to be one area with open problems. Related to this I think there are still open problems with regard to deriving tighter non-convex surrogate losses such that the issues of non-convexity does not overshadow the problems with inconsistency of convex unbounded loss functions. David McAllester has done some nice work in these areas recently, in particular for structured prediction. Another interesting open problem is whether to use averaging or not with stochastic gradient descent for strongly convex, but possibly non-smooth functions (like hinge loss). For more details see this paper by Omar Shamir (you can even get $50 if you solve this problem according to the paper :) |
