What happens when you increase the penalty strength alpha in ridge regression?
Options
- Coefficients grow, which lowers bias and raises variance
- Coefficients shrink, which raises bias and lowers variance (Correct option)
- Some coefficients become exactly zero
- Only the intercept changes
Answer
Coefficients shrink, which raises bias and lowers variance
Why
The penalty makes large coefficients expensive, so the model is pulled toward simpler fits. A simpler fit follows the training data less closely (more bias) but changes less between samples (less variance).
Too much penalty underfits; too little leaves overfitting. Choose alpha with cross-validation.
Ridge shrinks coefficients but does not zero them out; that is lasso (L1).
Remember
More regularization: more bias, less variance.