What is a good root mean squared (RMSE) value?

What is a good RMSE value? Simply explained

Root Mean Squared Error (RMSE) is a popular metric to use for regression machine learning models, but what is good value? In this post, I explain what RMSE is, what a good value is, and answer some common questions.

Stephen Allwright
Stephen Allwright

Root Mean Squared Error (RMSE) is a popular metric to use for regression machine learning models, but what is good value? In this post, I explain what RMSE is, what a good value is, and answer some common questions.

What is RMSE?

Root Mean Squared Error (RMSE) is the square root of the mean squared error between the predicted and actual values.

Squared error, also known as L2 loss, is a row-level error calculation where the difference between the prediction and the actual is squared. RMSE is the aggregated mean and subsequent square root of these errors, which helps us understand the model performance over the whole dataset.

A benefit of using RMSE is that the metric it produces is on the same scale as the unit being predicted. For example, calculating RMSE for a house price prediction model would give the error in terms of house price, which can help end users easily understand model performance.

The formula for calculating RMSE is:

formula for calculating root mean squared error (rmse)

How do you calculate RMSE?

To calculate RMSE for a set of predictions and their corresponding actual values, you need to follow these steps:

  1. Calculate the difference between each prediction and the actual value
  2. Square each of these values
  3. Calculate the mean of these squared values
  4. Calculate the square root of this mean

Example of calculating RMSE

Let’s look at an example where we calculate RMSE for a model which is predicting people’s height.

Actual Prediction Difference Squared Error
180 190 -10 100
160 155 5 25
170 172 -2 4
190 193 -3 9
200 196 4 16

RMSE = sqrt[(100 + 25 + 4 + 9 + 16)/5] = 5.55

Calculate RMSE in Python with Numpy

We can also calculate RMSE in Python using either the scikit-learn or Numpy packages. Here is an example of how this could be done using Numpy.

import numpy as np

actual = np.array([1100,2000,550,1510,2000])
prediction = np.array([1000,2000,500,1500,3000])

rmse = np.sqrt(np.mean((prediction-actual)**2))

Should RMSE be high or low?

RMSE is a metric which ranges from 0 to infinity, where the closer the score is to 0 the better performing the model is. So the RMSE value should be as low as possible.

How can I compare RMSE values from different models?

RMSE is an absolute error value, meaning that it is only relevant for the dataset and model it is calculated for. Therefore, RMSE cannot be used to compare model performance across different datasets. One way around this issue is to calculate the Root Mean Squared Percentage Error (RMSPE) which is the RMSE calculated as a percentage, much like MAPE.

What is a good RMSE value?

The closer RMSE is to 0, the more accurate the model is. But RMSE is returned on the same scale as the target you are predicting for and therefore there isn’t a general rule for what is considered a ‘good’ value. How good your metric value is can only be evaluated within the dataset context you are working.

Let’s try to unpack this more by looking at an example.

An RMSE of 1,000 for a house price prediction model is most likely seen as good because house prices tend to be over $100,000. However, the same RMSE of 1,000 for a height prediction model is terrible as the average height is around 175cm.

So unfortunately there is no standard for what a good value is, you will have to decide what is acceptable in the context of your project.


How to interpret RMSE

Metric calculators

RMSE calculator

Regression metrics

Mean Squared Error
Mean Absolute Error
Mean Absolute Percentage Error
Median Absolute Percentage Error
R2

Metric comparisons

RMSE vs MAE
RMSE vs MSE
RMSE vs MAPE

References

Numpy sqrt documentation
Sklearn MSE documentation

Metrics

Stephen Allwright Twitter

I'm a Data Scientist currently working for Oda, an online grocery retailer, in Oslo, Norway. These posts are my way of sharing some of the tips and tricks I've picked up along the way.