What is MDAPE and how do I calculate it in Python?

What is MDAPE?

Median Absolute Percentage Error (MDAPE) is an error metric used to measure the performance of regression machine learning models. It outputs the error as a percentage, making it a popular metric to use as the resulting error can be easily understood by all end users without added context.

What's the difference between MDAPE and MAPE?

MDAPE and MAPE are both error metrics used for assessing regression machine learning models, and both use a very similar calculation. The difference lies in that MDAPE returns the median value of all the errors, whereas MAPE returns the mean. This is a minor calculation difference but it can have a big impact on your result. The result of this change you will notice is that MAPE is more sensitive to outliers than MDAPE, so if this is important for your use case MDAPE could be the metric of choice for you.

What is a good MDAPE value?

MDAPE returns error as a percentage, making it easy for end users to understand the performance. The lower the percentage, the more accurate the model, and vice versa. It goes without saying that how good your MDAPE score is depends very much on your use case, but a general rule of thumb that I follow is:

MDAPEInterpretation
< 10 %Very good
10 % - 20 %Good
20 % - 50 %OK
> 50 %Not good

Calculate MDAPE in Python

Unlike other popular metrics for machine learning models, MDAPE is not available through the scikit-learn package. Therefore we need to create the calculation ourselves using the numpy package. The following code shows you how:

import numpy as np

actual = [100,90,110,150]
predicted = [110,100,90,145]

mdape = np.median((np.abs(np.subtract(actual, predicted)/ actual))) * 100

Positives and negatives of using MDAPE

MDAPE is a useful error metric, however there are upsides and downsides to using it in your solution. These are:

Positives

  • Error is returned as a percentage, making it easy for end users to understand
  • Percentage error makes it possible to compare performance with models trained on different datasets
  • Not as sensitive to outliers as MAPE

Negatives

  • If your actual values can be 0, then MDAPE won't be possible to calculate

Calculate MAPE with 0 values
MSE score
RMSE score
MAE score
R2 score

References

Numpy documentation

Stephen Allwright

Stephen Allwright

I'm a Data Scientist currently working for Oda, an online grocery retailer, in Oslo, Norway. These posts are my way of sharing some of the tips and tricks I've picked up along the way.
Oslo, Norway