What is a good MAPE score?
MAPE (Mean Absolute Percentage Error) is a common regression machine learning metric, but it can be confusing to know what a good score actually is. In this post, I explain what MAPE is, what a good score is, and answer some common questions that people have.
MAPE (Mean Absolute Percentage Error) is a common regression machine learning metric, but it can be confusing to know what a good score actually is. In this post, I explain what MAPE is, what a good score is, and answer some common questions that people have.
What is MAPE?
Mean Absolute Percentage Error (MAPE) is the mean of all absolute percentage errors between the predicted and actual values.
It is a popular metric to use as it returns the error as a percentage, making it both easy for end users to understand and simple to compare model accuracy across use cases and datasets.
Mathematical formula for MAPE
The formula for calculating MAPE is as follows:
This formula helps us understand one of the important caveats when using MAPE. In order to calculate this metric, we need to divide the difference by the actual value. This means that if you have actual values close to or at 0 then your MAPE score will either receive a division by 0 error, or be extremely large. Therefore, it is advised to not use MAPE when you have actual values close to 0.
Is MAPE similar to MAE?
MAPE is similar to MAE but it goes one step further, by adding in the actual value division to convert it to a percentage. This is not to say that MAPE is better than MAE, as they offer different views on the error of your model and are both good metrics to track in your model development.
Positives and negatives of using MAPE
MAPE is a popular metric to use for regression models, however, there are some things you must consider when optimising for this metric:
Positives of using MAPE as a metric
- Easy for end users to understand as the error is a percentage
- Possible to compare model accuracy across datasets and use cases
- Easily implemented in Python
Negatives of using MAPE as a metric
- Not possible to use when actual values can be at or close to zero
When to use MAPE
MAPE should be used when either communicating results to end users is important or when you need to be able to compare your results with other models. It shouldn’t be used when you have actual values that are close to or at zero due to the division by zero error.
How to calculate MAPE in Python
Calculating MAPE in Python is simple to do using the scikit-learn package, below is a simple example showing how to implement it:
from sklearn.metrics import mean_absolute_percentage_error
actual = [10,12,8]
prediction = [9,14.5,8.2]
mape = mean_absolute_percentage_error(actual, prediction)
What is a good MAPE score?
MAPE returns error as a percentage, making it easy to understand the 'goodness' of the error value. It goes without saying that how 'good' your MAPE score is depends on your use case and dataset, but a general rule of thumb that I follow is:
| MAPE | Interpretation |
|---|---|
| < 10 % | Very good |
| 10 % - 20 % | Good |
| 20 % - 50 % | OK |
| > 50 % | Not good |
How do you compare MAPE values?
MAPE is returned as a percentage, which makes it possible to compare values across different datasets and models. No adjustments are required to compare these values, but be aware that different use cases may have different understandings of what a ‘good’ value is.
What is a good MAPE for forecasting?
A good value is dependent upon your use case, but in general, a MAPE lower than 20% is considered good for time series forecasting. This would indicate that on average the forecasts over the whole time period were less than 20% away from the actual values.
Can MAPE be more than 100%?
MAPE values can be more than 100% as predictions can have an error which is greater than the actual value.
Is a higher or lower MAPE better?
MAPE is a percentage error metric where the value corresponds to the average amount of error that predictions have. Therefore, a lower MAPE is better, where the lower the value the more accurate the model is.
What is the MAPE value range?
MAPE values range from 0 to infinity, where the lower the value the more accurate the predictions are.
What value of MAPE is acceptable?
A good score is dependent upon your use case, but in general, a MAPE value lower than 20% is considered a good value.
Related articles
Metric calculators
Regression metrics
What's a good MSE value?
What's a good RMSE value?
How to use MDAPE
What's a good R-Squared value?
Negative percent error
Interpretation of MAPE scores
Metric comparisons
RMSE vs MAPE, which is the best regression metric?
MAE vs MAPE, which is best?
References
