|
I have been looking at the results of some consensus clustering methods and I am impressed by the benefits of a consensus made on the results of using a single algorithm. I do not find a lot of discussion on the theoretical merits of using a consensus though. There is a lot of good discussion for why kmeans fails on not spherial/convex shapes of clusters for example, but the theoretical benefits for a consensus is still a mystery to me. Can someone explain what advantages a consensus is giving? |
|
Consensus clustering is essentially bagging applied to clustering and the benefits of consensus clustering are essentially the same as the benefits of bagged classification & regression. It reduces artifacts caused by sampling fluctuation (ie variance error), since it averages over perturbed versions of the training sample. An excellent explanation of bagging in the supervised setting is a talk by Philip Kegelmeyer, (his other papers & talks are also very instructive). K-means probably works particularly well as the base algorith for consensus clustering for the same reason that trees work well with bagging in the supervised setting, namely that k-means is an unstable algorithm (moving, adding or deleting a single point can have a significant impact on the final clustering). This leads to diversity among the resampled clusterings, which is the key to the success of ensemble methods. From my brief reading (I'd be happy to be shown to be wrong), it feels like it's just using half of the ideas from bagging. It uses the averaging many peers idea from bagging (and it's interesting because it's much harder to average a clustering than a classification), but not the random sample from input to form new distribution idea.
(Sep 20 '11 at 11:06)
Rob Renaud
1
In [http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.88.263] Monti et al specifically talk about resampling & a "Resample" procedure is a parameter of the algorithm in figure 1.
(Sep 20 '11 at 11:48)
Daniel Mahler
|
