Since MRP works primarily as a push system, accuracy of the input data
is the number one priority of the Planner. Forecasts that basically build and
drive the Master Production Schedule, should depict the actual demand the
closest possible. Inaccuracy will always lead to loss of money, either due to
holding excessive stock or because of the lost sales.
The basic and most logical way to measure accuracy is by measuring the
deviation between the forecasted demand and the actual demand. The difference
between these two numbers, when is expressed as a percentage, is the actual
forecast error.
Sample
SKU
|
|
Forecasted
Demand
|
80
|
Actual
Demand
|
75
|
Forecast Error (A-F/A)
|
-6,67%
|
Abs
of Error
|
6,67%
|
Accuracy [1-abs(Error)]
|
93,33%
|
Most information I've found, propose to use the forecast error as an
absolute number. I prefer to use as a signed percentage, because it clearly
shows the direction of the deviation as well as the deviation itself. So, a
negative forecast error shows clearly that the actual demand was less than what
was forecasted and a positive forecast error would mean the opposite. Accuracy
on the other hand is just 1 – the absolute value of the forecast error. In case
that the forecast error is more than 100% then accuracy would just have a zero
value, since there is no point showing negative accuracy.
Although the above calculations are pretty much straight forward, things
are a little more complicate when it comes to measure the accuracy for many
SKUs and thus the whole forecast. In order to achieve this, it is necessary to
calculate what is called the Mean Absolute Percentage Error or else MAPE, which
is basically the Average Absolute Error divided by the Average Actual Quantity.
SKU
A
|
SKU
B
|
SKU
C
|
SKU
D
|
Total
|
||
Forecasted
Demand
|
120
|
50
|
200
|
500
|
870
|
|
Actual
Demand
|
100
|
40
|
51
|
450
|
641
|
|
Absolute
Error
|
20
|
10
|
149
|
50
|
229
|
|
Average
ABS Error (MAE)
|
57,25
|
|||||
Average
Actual Demand
|
160,25
|
|||||
MAPE
|
35,73%
|
|||||
Forecast
Accuracy (1-MAPE)
|
64,27%
|
|||||
From what I know, MAPE is the method most
commonly used today. It is easy to calculate and self explicable. The only
thing I don’t like about it is the use of the absolute values. I don’t know how
long back in time MAPE can be traced but an engineer would definitely decide to
solve the problem with a statistical approach. What else could it be better
than trying to calculate the Standard Deviation of the Error? Well this
approach is called RMSE, meaning the Root of the Mean Squared Error and has
no difficulty at all, but still many people avoid it.
If you had a sample that you needed to find the
standard deviation, you would simply calculate the mean, subtract the mean from
each sample value and square the results. The square root of the average of the
squared values would be the standard deviation. The same applies here but instead
of calculating the mean of the samples we will use the Actual Demand as a
reference point.
SKU A
|
SKU B
|
SKU C
|
SKU D
|
Total
|
||
Forecasted Demand
|
120
|
50
|
200
|
500
|
870
|
|
Actual Demand
|
100
|
40
|
51
|
450
|
641
|
|
Squared Error
|
400
|
100
|
22201
|
2500
|
25201
|
|
Average of Squared
Errors
|
6.300,25
|
|||||
RMSE
|
79,37
|
|||||
Although for me this makes much more sense well
it misses something that most would ask for. This is the % sign. RMSE is an
absolute number such as the standard deviation and that’s why many people
choose MAPE. I choose to calculate both
MAPE and RMSE as accuracy indicators.
From this point you can do many more tricks.
The cleverest one would be to use weights for the simple MAPE. If the weigh is
the standard cost, then the most expensive SKU’s would weigh more and would
adjust the MAPE to a more logical value. So, the accuracy error of the cheap
SKU’s would be less significant than the error of the expensive SKU’s.
Once you automate the calculations in an Excel
spreadsheet then there is even a better solution. Use all the above!
