topic Re: Why does proc arima with NoEst throw 'There is not enough data to fit the model' error? in SAS Forecasting and Econometrics

Why does proc arima with NoEst throw 'There is not enough data to fit the model' error?

mark-adamson — Tue, 16 Aug 2016 14:39:47 GMT

I am using proc arima in SAS 9.4 to produce a forecast using a previously calibrated model, but it is throwing an error as if it is trying to calibrate the model itself :

ERROR: There is not enough data to fit the model

sample data:

data inputs;
input x var1 var2 var3 var4 var5;
datalines;
  20    5   2   4   5   4
  25    12  56  13  44  4
  20    5   2   4   5   4
  25    12  56  13  44  4
  20    5   2   4   5   4
  25    12  56  13  44  4
   .    2   5   6   5   4
;

failing version:

proc arima;
  identify 
    data = inputs 
    var = x
    crossCorr = ( var1 var2 var3 var4 var5 ) 
    noPrint;

  estimate 
    p = 1 input = ( var1 var2 var3 var4 var5 )
    ar = 0.9
    initVal = ( 0.1$var1 0.2$var2 0.3$var3 0.4$var4 0.4$var5 ) 
    noint 
    noEst /* Using noEst so should not need to do any estimation and short data-set should not be a problem */
    method=ml 
    noprint
;

  forecast lead=1 out=outputs noOutAll noprint;
quit;

If I remove the final variable from the model, it works fine:

proc arima;
  identify 
    data = inputs 
    var = x
    crossCorr = ( var1 var2 var3 var4 ) 
    noPrint;

  estimate 
    p = 1 input = ( var1 var2 var3 var4 )
    ar = 0.9
    initVal = ( 0.1$var1 0.2$var2 0.3$var3 0.4$var4 ) 
    noint 
    noEst /* Using noEst so should not need to do any estimation and short data-set should not be a problem */
    method=ml 
    noprint 
;

  forecast lead=1 out=outputs noOutAll noprint;
quit;

I can also get it to 'work' by adding one more value to the data. However, this shouldn't be necessary when the model is already calibrated (using much more data).

I've checked the SAS documentation to see if there are any flags to prevent the unnecessary check that causes this error but none of them helped.

Cross-posted on stack-overflow

Re: Why does proc arima with NoEst throw 'There is not enough data to fit the model' error?

rselukar — Wed, 17 Aug 2016 19:14:53 GMT

ARIMA does exit if the number of non-missing observations are less than or equal to the number of parameters in the model whether parameters are known or not. This behavior is known. This use case scenario is somewhat uncommon and ARIMA does not handle it. As you have said, you could add some rows (possibly artificial data) at the beginning to take care of this scenario. By the way, for these types of scenarios Forecast Server offers specialized scoring functionality for ARIMA models that could be of interest to you.

Re: Why does proc arima with NoEst throw 'There is not enough data to fit the model' error?

mark-adamson — Thu, 18 Aug 2016 07:32:16 GMT

Thank you for the quick response and clear answer. That is unfortunate and it means we will go back to R for this model which handles it without any problems. Adding in artificial data will change the result in our real model since we are using MA terms, so that isn't a viable workaround for us.

Re: Why does proc arima with NoEst throw 'There is not enough data to fit the model' error?

rselukar — Thu, 18 Aug 2016 11:52:10 GMT

We cannot let you go back to R without some fight (just kidding)! You can use PROC SSM (as shown below) to get what you want. This procedure is for linear state space modeling (ARIMA models are state space models). See the doc at http://support.sas.com/documentation/cdl/en/etsug/68148/HTML/default/viewer.htm#etsug_ssm_syntax.htm

In the code illustration below I have added an MA(1) term to the model since you said your models might have MA terms. Moreover, you can also specify the error variance from your calibrated model.

data inputs;

input x var1 var2 var3 var4 var5;

datalines;

20 5 2 4 5 4

25 12 56 13 44 4

20 5 2 4 5 4

25 12 56 13 44 4

20 5 2 4 5 4

25 12 56 13 44 4

. 2 5 6 5 4

;

/* Without known AR error variance*/

proc ssm data=inputs;

trend ar(arma(p=1 q=1)) ar=0.9 ma=0.3;

cin = 0.1*var1 + 0.2*var2 + 0.3*var3 + 0.4*var4 + 0.4*var5;

state tf(1) sinput=(cin);

comp tfTerm = tf[1];

model x = tfTerm ar / print=smooth;

output out=for pdv;

run;

/* With known AR error variance=10 say*/

proc ssm data=inputs;

trend ar(arma(p=1 q=1)) ar=0.9 ma=0.3 variance=10;

cin = 0.1*var1 + 0.2*var2 + 0.3*var3 + 0.4*var4 + 0.4*var5;

state tf(1) sinput=(cin);

comp tfTerm = tf[1];

model x = tfTerm ar / print=smooth;

output out=for pdv;

run;

Re: Why does proc arima with NoEst throw 'There is not enough data to fit the model' error?

mark-adamson — Thu, 18 Aug 2016 12:28:10 GMT

Thanks for that, proc ssm looks very promising and powerful. I may have a go at converting what I have done with proc arima to use it. It looks like it has nicer syntax than proc arima for doing differencing and including seasonality too, both relevant to us.

SAS gets +5 points in the SAS vs. R showdown...

Re: Why does proc arima with NoEst throw 'There is not enough data to fit the model' error?

rselukar — Thu, 18 Aug 2016 12:49:36 GMT

The syntax of PROC SSM is more complex than ARIMA and you might need some time to get used to it. If you need syntax help for converting any ARIMA spec (including transfer function type terms) contact me separately and I will help you.