Bias/ Variance
Vocabulary
Bias: The bias of a model is the difference between the expected prediction and the correct model that we try to predict for given data points.
Variance: The variance of a model is the variability of the model prediction for given points.
Bias/ variance tradeoff: The simpler the model, the higher the bias, and more complex the model, the higher the variance.
| Underfitting | Just right | Overfitting | |
|---|---|---|---|
| Symptoms | High training error, training error close to test error, high bias | Training error slightly lower than test error | Very low training error, training error much lower than test error, high variance |
| Regression illustration |  |  |  |
| Classification illustration |  |  |  |
| Deep learning illustration |  |  |  |
| Possible remedies | Complexify model, Add more features, Train longer | . | Regualarization, Get more data |