Explain the bias-variance trade-off and how you react to each symptom.

Bias: error from a model too simple for the pattern. Symptom: high error on training and validation data (underfitting).
Variance: error from a model too sensitive to the training sample. Symptom: low training error and much higher validation error (overfitting).
More model capacity lowers bias and raises variance; the best model sits in between.
High bias: richer features, a more flexible model, less regularization.
High variance: more data, a simpler model, stronger regularization, fewer features, bagging or early stopping.
Plot learning curves (error against training size) to see which case you are in.