Knowing what is a good MSE score is a common question amongst data scientists. The closer your MSE score is to 0, the better performing and more accurate the model is. However, translating that score into an understanding of 'goodness' in your use case, and also comparing this amongst other models, is unfortunately more of an art than a science. In this post I seek to explain simply how to work with MSE and how to know what a good MSE score is.
What is a good MSE score?
Mean squared error (MSE) is a metric for assessing the performance of regression machine learning models. It is a common metric to use when assessing how accurate your model is performing and how this has changed over time.
MSE is often discussed alongside the more popular regression metric, RMSE. This is because RMSE is a continuation of MSE, in that it is calculated by taking the square root of MSE.
Calculate MSE in Python with sklearn
MSE is an incredibly simple metric to calculate. It's produced by taking the mean of all the squared errors in your dataset. If you are using Python it is easily implemented by using the scikit-learn package. An example can be seen here:
from sklearn.metrics import mean_squared_error y_true = [10, -5, 4, 15] y_pred = [8, -1, 5, 13] mse = mean_squared_error(y_true, y_pred)
What is a good MSE score and how do I interpret it?
The closer your MSE score is to 0, the more accurate and better performing your model is. Unfortunately, there is no 'good' score for MSE, it can only be used to say whether the model has become better or worse at predicting for your target than the previous attempt.
An important aspect of MSE to note is that the error metric is not returned in the same scale as the values you are predicting for. For example; if you are predicting house prices and your MSE is 10,000, this does not mean that your model is accurate within $10,000 of a house price. Because of this I would suggest using MSE alongside other metrics, such as MAE or RMSE, which provide error metrics in the same scale as your target.
Should I use MSE as a metric?
It is common to track multiple metrics when developing a machine learning model, this is to ensure that the performance of the model is understood from multiple angles. MSE, however, is often included as a metric of lower importance in these, which is down to heavily weighted ratio of disadvantages to advantages it has:
Advantages of using MSE
- Easy to implement and understand
Disadvantages of using MSE
- Error value not given in terms of the target
- Difficult to understand the 'goodness' of score
- Not comparable across use cases
- Sensitive to outliers (this could equally be a positive as well)