Which model is most affected if you forget to scale numeric features that have very different ranges?
Options
- Random forest
- Gradient-boosted trees
- Decision tree
- k-nearest neighbours (Correct option)
Answer
k-nearest neighbours
Why
k-nearest neighbours compares distances. A feature measured in thousands (salary) swamps one measured in units (number of children), so scaling changes which points are neighbours.
Tree-based models split on thresholds one feature at a time, so a monotonic rescaling does not change their splits.
SVMs, linear models with regularization and neural networks also need scaling.
Remember
Distance and penalty methods need scaling; trees do not.