What helps a random forest generalize better than one deep decision tree?
Options
- It builds trees one after another, each fixing the mistakes of the previous one
- It prunes the tree with gradient descent
- It trains many trees on bootstrap samples with random subsets of features and averages their votes, which lowers variance (Correct option)
- It uses more training data than a single tree
Answer
It trains many trees on bootstrap samples with random subsets of features and averages their votes, which lowers variance
Why
The trees in a random forest are trained independently, each on a bootstrap sample of the rows and with a random subset of features at each split. Their errors are less correlated, and averaging cancels much of the noise.
Building trees one after another describes boosting, not a random forest: there the trees are built in sequence and each one learns from the errors of the ensemble so far.
A forest keeps the low bias of deep trees but reduces their variance.
Remember
Forest = bagging + random features: lower variance.