What is a good MSE score? Mean Squared Error explained!

Knowing what is a good MSE score is a common question amongst data scientists. The closer your MSE score is to 0, the better performing and more accurate the model is. However, translating that score into an understanding of 'goodness' in your use case, and also comparing this amongst other models, is unfortunately more of an art than a science. In this post I seek to explain simply how to work with MSE and how to know what a good MSE score is.

What is a good MSE score?

Mean squared error (MSE) is a metric for assessing the performance of regression machine learning models. It is a common metric to use when assessing how accurate your model is performing and how this has changed over time.

MSE is often discussed alongside the more popular regression metric, RMSE. This is because RMSE is a continuation of MSE, in that it is calculated by taking the square root of MSE.

Calculate MSE in Python with sklearn

MSE is an incredibly simple metric to calculate. It's produced by taking the mean of all the squared errors in your dataset. If you are using Python it is easily implemented by using the scikit-learn package. An example can be seen here:

from sklearn.metrics import mean_squared_error

y_true = [10, -5, 4, 15]
y_pred = [8, -1, 5, 13]

mse = mean_squared_error(y_true, y_pred)

What is a good MSE score and how do I interpret it?

The closer your MSE score is to 0, the more accurate and better performing your model is. Unfortunately, there is no 'good' score for MSE, it can only be used to say whether the model has become better or worse at predicting for your target than the previous attempt.

An important aspect of MSE to note is that the error metric is not returned in the same scale as the values you are predicting for. For example; if you are predicting house prices and your MSE is 10,000, this does not mean that your model is accurate within $10,000 of a house price. Because of this I would suggest using MSE alongside other metrics, such as MAE or RMSE, which provide error metrics in the same scale as your target.

Should I use MSE as a metric?

It is common to track multiple metrics when developing a machine learning model, this is to ensure that the performance of the model is understood from multiple angles. MSE, however, is often included as a metric of lower importance in these, which is down to heavily weighted ratio of disadvantages to advantages it has:

Advantages of using MSE

  1. Easy to implement and understand

Disadvantages of using MSE

  1. Error value not given in terms of the target
  2. Difficult to understand the 'goodness' of score
  3. Not comparable across use cases
  4. Sensitive to outliers (this could equally be a positive as well)

Regression metrics

MSE calculator
What is a good MAE score?
What is a good R2 score?
What is a good MAPE score?
What is MDAPE and how do I calculate it in Python?

Regression metric comparisons

RMSE vs MSE, which should I use?
MSE vs MAE, which is the better regression metric?

References

Scitkit-learn documentation


Stephen Allwright

Stephen Allwright

I'm a Data Scientist currently working for Oda, an online grocery retailer, in Oslo, Norway. These posts are my way of sharing some of the tips and tricks I've picked up along the way.
Oslo, Norway