What is the difference between a model's parameters and its hyperparameters?

Options

1. Parameters are set by hand before training; hyperparameters are learned from the data
2. There is no difference; the two words mean the same
3. Parameters belong to regression models and hyperparameters to classification models
4. Parameters are learned from the training data; hyperparameters are chosen before training starts (Correct option)

Answer

Parameters are learned from the training data; hyperparameters are chosen before training starts

Why

Parameters are the numbers the model learns while training, such as the weights of a linear model or the split values of a tree.

Hyperparameters are settings you choose before training, such as the learning rate, the maximum depth of a tree or the strength of regularization.

You find good hyperparameters by trying values and measuring on validation data, not by training them.

Remember

Parameters are learned; hyperparameters are chosen.

Key terms

• Model parameter
• Hyperparameter
• Validation set