BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
Edoedoedo
Pyrite | Level 9

Hi, I have a time series with it's prediction (attached). I calculate the SMAPE with:

 

cas session;

libname D cas caslib="CASUSER";

proc tsmodel 
    data=D.TESTSMAPE
    outobj=(utlstat=D.OUTSTATS(replace=YES));
    id DATE interval=day;
    var MEASURED SARIMA;
    require utl;
    submit;
    declare object utlstat(utlstat);
    rc = utlstat.Collect(MEASURED, SARIMA);
    endsubmit;
run;

In D.OUTSTATS I get that SMAPE = 719.77. That is very wierd, since the SMAPE metric is bounded between [0%,200%].

 

I tried the same calculation by hand using the standard SMAPE formula

 

100%/N * SUM [ |PREDICTED-MEASURED| / ((|MEASURED|+|PREDICTED|)/2) ]

 

 and I get SMAPE = 119.43 which seems correct, as shown in the attached excel file.

 

So what is SAS doing? Where that 719.77 value comes from?!?

 

Thank you

Regards

1 ACCEPTED SOLUTION

Accepted Solutions
SASCom1
SAS Employee

Since you have opened Technical Support track regarding this question, I am posting below a summary of the response you already received from Technical Support just to close out this thread.

 

The TSMODEL Procedure calculates the absolute symmetric percent error(ASPE), and its mean, as originally proposed by Markakis and Armstrong.  From the Statistics of Fit section of the documentation, the formula for ASPE is:

 

| 100 * ( SARIMA - MEASURED) / 0.5 * ( SARIMA + MEASURED) |

 

Rob Hyndman wrote an interesting article on this topic, Errors on percentage errors | Rob J Hyndman, in which he credits the wikipedia formula to Chen and Yang (2004). 

While we don't have a copy of the Chen and Yang paper on hand to verify, R&D is considering adding the statistic to those provided by TSMODEL in the future.

View solution in original post

3 REPLIES 3
sbxkoenk
SAS Super FREQ

Hello,

 

If you don't get a satisfactory answer on this very quick I would contact Technical Support in your local SAS office.
If what you report is really a problem in the software it will correctly and quickly stream through to R&D and it will be fixed.

Whatever you are doing wrong (or not), a value of > 700 should never be reported if the symmetric mean absolute percentage error (SMAPE or sMAPE) is bounded between 0% and 200%.

 

Do you have missing values in the series at the beginning of the data set? I remember about a problem with SMAPE and missing values at the beginning of the series but it was fixed in 2005!

 

Thanks,

Koen

Edoedoedo
Pyrite | Level 9

Thank you, as you suggested I opened a ticket for Technical Support.

I thought as well that the problem may have something related to missing values, but after removing all missing values still the result is 719, so I guess the missing are handled correctly and the problem is not there.

 

I'll share useful news as soon as I have ones.

 

Regards

SASCom1
SAS Employee

Since you have opened Technical Support track regarding this question, I am posting below a summary of the response you already received from Technical Support just to close out this thread.

 

The TSMODEL Procedure calculates the absolute symmetric percent error(ASPE), and its mean, as originally proposed by Markakis and Armstrong.  From the Statistics of Fit section of the documentation, the formula for ASPE is:

 

| 100 * ( SARIMA - MEASURED) / 0.5 * ( SARIMA + MEASURED) |

 

Rob Hyndman wrote an interesting article on this topic, Errors on percentage errors | Rob J Hyndman, in which he credits the wikipedia formula to Chen and Yang (2004). 

While we don't have a copy of the Chen and Yang paper on hand to verify, R&D is considering adding the statistic to those provided by TSMODEL in the future.

sas-innovate-2024.png

 

Secure your spot at the must-attend AI and analytics event of 2024: SAS Innovate 2024! Get ready for a jam-packed agenda featuring workshops, super demos, breakout sessions, roundtables, inspiring keynotes and incredible networking events.

 

Register by March 1 to snag the Early Bird rate of just $695! Don't miss out on this exclusive offer. 

 

Register now!

Multiple Linear Regression in SAS

Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 3 replies
  • 607 views
  • 2 likes
  • 3 in conversation