topic Re: How to extrapolate or forecast values based on a fitted curve of existing values in SAS Forecasting and Econometrics

How to extrapolate or forecast values based on a fitted curve of existing values

ANevola — Thu, 12 Jul 2018 02:03:47 GMT

I'd like to extrapolate missing values based on a fitted curve of existing data points, using version 9.4. Here's my data:

Year NPs_per_pop
2008 .
2009 .
2010 0.3624
2011 0.3971
2012 0.4366
2013 0.4804
2014 0.5291
2015 0.5859

Graphing the data looks like this:

I'd like to estimate 2008 and 2009 based on a fitted curve of the existing 2010-2015 values. Since I couldn't figure out how to "forecast" into the past, I first reversed the order of the data set, so that the first observation is from 2015 and the last is from 2008:

Order NPs_per_pop
1 0.5859
2 0.5291
3 0.4804
4 0.4366
5 0.3971
6 0.3624
7 .
8 .

The closest I've been able to come is estimating based on the straight line of best fit, using proc esm:

proc esm data=original out=estimated lead=2;
	forecast NPs_per_pop / model=linear;
run;

which produces this (when the order is reversed again, so it goes from 2008-2015):

The linear model is an ok fit, but not great..... one reason is that the values should always be positive, fitting more of an exponential model. I tried the different options of proc esm (e.g., transform=logistic), but nothing seemed to populate 2008 and 2009 with a fitted curve. Any help on how to do that would be appreciated!

Re: How to extrapolate or forecast values based on a fitted curve of existing values

Ksharp — Thu, 12 Jul 2018 13:24:25 GMT

You could try PROC LOESS

proc reg data=have;

model y= x x^2 ;

quit;

@Rick_SAS wrote a blog about it before .

Re: How to extrapolate or forecast values based on a fitted curve of existing values

ANevola — Thu, 12 Jul 2018 15:50:48 GMT

Thanks for you response.

Proc reg gave me an error under the exponent operator (**), saying "ERROR 22-322: Syntax error, expecting one of the following: a name, ;, -, /,:, _ALL_, _CHARACTER_, _CHAR_, _NUMERIC_, {."

proc reg data=reg;
	model NPs_per_pop = year year**2;
quit;

I tried the syntax of proc loess in one of the examples given, but it wouldn't produce a smoothed plot of my variables:

ods graphics on;
proc loess data=reg;
ods output OutputStatistics = Fit
                 FitSummary=Summary;
model NPs_per_pop = year / degree=2 select=AICC(steps) smooth = 0.6 1.0
                      direct alpha=.01;
run;
ods graphics off;

Either way, though, neither procedure seems to produce an extrapolation of missing data points. Could you give a little more detail on how you were thinking those procedures would fill in missing values based on a fitted curve?

Re: How to extrapolate or forecast values based on a fitted curve of existing values

rselukar — Thu, 12 Jul 2018 19:53:54 GMT

Like PROC ESM, PROC SSM is also part of SAS/ETS. You could use it for model based back-propagation, interpolation and forecasting. It is a bit more involved than ESM. Anyway, here is one possibility:

data test;
input year y@@;
year = mdy(1,1, year);
format year date.;
datalines;
2008    .
2009    .
2010    0.3624
2011    0.3971
2012    0.4366
2013    0.4804
2014    0.5291
2015    0.5859
;
proc ssm data=test;
   id year interval=year;
   trend curve(ps(2));
   irregular wn;
   model y = curve wn;
   output out=for press;
run;
proc sgplot data=for;
   scatter x=year y=y;
   series x=year y=smoothed_curve;
   reg x=year y=y;
run;

See the attached fit.

Re: How to extrapolate or forecast values based on a fitted curve of existing values

Ksharp — Fri, 13 Jul 2018 11:51:24 GMT

data test;
input year y@@;
datalines;
2008    .
2009    .
2010    0.3624
2011    0.3971
2012    0.4366
2013    0.4804
2014    0.5291
2015    0.5859
;

ods output sgplot=temp;
proc sgplot data=test;
reg x=year y=y/cli clm degree=2;
run;

proc print noobs;run;

Re: How to extrapolate or forecast values based on a fitted curve of existing values

ANevola — Fri, 13 Jul 2018 15:24:58 GMT

Thanks for your response. It looks like my computer doesn't have sufficient memory for proc ssm, though. I got the following error message:

8485  proc ssm data=reg;
8486     id year interval=year;
8487     trend curve(ps(2));
8488     irregular wn;
8489     model NPs_per_pop = curve wn;
8490     output out=for press;
8491  run;

ERROR: Insufficient memory for data reading.

I'll try to find a computer with more memory to run the code you suggested. Thanks again.

Re: How to extrapolate or forecast values based on a fitted curve of existing values

rselukar — Fri, 13 Jul 2018 15:34:21 GMT

That is very strange. SSM should not have memory issues for such a small sized problem even on a basic computer. Anyway, keep me posted.

Re: How to extrapolate or forecast values based on a fitted curve of existing values

rselukar — Fri, 13 Jul 2018 15:52:38 GMT

The SSM procedure handles a variety of models and it's syntax and output might take some getting used to. You can see Example 3 ("Backcasting, Forecasting, and Interpolation") in the SSM doc for an additional example. If you just want the back-casted values in your case, you can use the following modification of the code (print=smooth option in the MODEL statement):

proc ssm data=test;
   id year interval=year;
   trend curve(ps(2));
   irregular wn;
   model y = curve wn / print=smooth;
   output out=for press;
run;