|
When creating decision trees in random forest, what happens if we select only the most informative attributes? Does it improve the performance of random forest? |
|
Generally with ensemble methods, there is a tradeoff between diversity and accuracy in each ensemble member. In the limit, you could imagine making every ensemble member as accurate as possible, but then they'd all be the same, and you'd actually get no advantage from adding extra trees. On the other hand, if all the ensembles all got the correct answer only 60% of the time, but in a way that was independent of every other tree, in the limit of infinite trees, the ensemble would always be correct. With just 15 trees, you'd be at 96% accuracy. The reason that Random Forests select the best feature from a random subset of features at each node is so that different trees can be very different from each other. It's really a tradeoff between the accuracy and diversity of the tree members. |
