What is a good MAE score?

What is Mean Absolute Error (MAE)?

MAE (Mean absolute error) is the average absolute (absolute means the non negative) error between actual and predicted values, and is a common metric to use for regression machine learning models. But how do you know what a good MAE value is? And how can one compare MAE values across different use cases?

What is a good MAE value?

Much like when using RMSE as an error metric, MAE is domain specific. Meaning that if you are predicting the price of a house, the error will be given in terms of house prices, however if you are predicting how many orders a customer will make next year, the MAE will be given in terms of orders. This makes it difficult to compare model accuracy across domains using MAE, as an MAE of 1,000 for a house price prediction would be an accurate model, however for predicting customer orders this would be terrible.

How to calculate MAE

Calculating MAE is simple to do manually, however most data scientists tend to use the package provided by scikit-learn for ease of use.

from sklearn.metrics import mean_absolute_error

actual = [100,120,80,110]
predicted = [90,120,50,140]

mae = mean_absolute_error(actual, predicted)

How can on compare MAE across different use cases?

MAE on it's own cannot be compared across use cases, however by converting this error to a percentage it is then understandable outside of the domain. This error is called Mean Absolute Percentage Error (MAPE) and is calculated like so:

from sklearn.metrics import mean_absolute_percentage_error

actual = [100,120,80,110]
predicted = [90,120,50,140]

mape = mean_absolute_percentage_error(actual, predicted)

Positives and negatives of using MAE

Like with any metric, there are positives and negatives to using MAE as an error metric to optimise for.

Positives

  1. Error is understandable for end users
  2. Value is given in terms of the domain you are working within

Negatives

  1. Sensitive to outliers
  2. Unless converting to MAPE, it is not comparable across domains

Regression metrics

Mean Squared Error (MSE)
R2 score
Median Absolute Percentage Error (MDAPE)

Metric comparisons

RMSE vs MAE, which should I use?
MAE vs MAPE, which is best?
MSE vs MAE, which is the better regression metric?

Metric calculators

MAE calculator

References

Sklearn documentation for MAE

Stephen Allwright

Stephen Allwright

I'm a Data Scientist currently working for Oda, an online grocery retailer, in Oslo, Norway. These posts are my way of sharing some of the tips and tricks I've picked up along the way.
Oslo, Norway