Accessing standardized residuals: arima

[sasalex2024]

Dear SAS Community,

Suppose I have standard PROC ARIMA commands like this:

proc arima data=a;

identify var=Yt crosscorr=(X1t X2t) noprint;

estimate q=1 input=( (1) X1t (1) X2t ) method=ml;

run;

The command displays a QQ plot of standardized residuals. However, if I want to access the values of those residuals, could you please guide me on how to do it?

I have tried using the outmodel etc, but I couldn't find a way to extract the residuals.

Thank you!

[sbxkoenk]

The below code provides you with the (X,Y)-coordinates of the markers in the QQ-plot (see data set named "work.abc"), but the data are sorted. I don't know how you can find out which observation belongs to which time stamp in your time series.

title1 'Simulated IMA(1,1) Series';
data a;
  u1 = 0.9; a1 = 0;
  do i = -50 to 100;
     a = rannor( 32565 );
     u = u1 + a - .8 * a1;
     if i > 0 then output;
     a1 = a;
     u1 = u;
  end;
run;

*ods trace on; ods trace off;
ODS output ResidualNormalityPanel=work.abc;
/*-- Simulated IMA Model --*/
proc arima data=a;
  identify var=u;
  run;
  identify var=u(1);
  run;
  estimate q=1 ;
  run;
quit;
/* end of program */

Ciao, Koen

[sasalex2024]

Dear Koen,

Thank you very much for your response. Your code works perfectly for the example model you provided. However, I have been unable to adapt it to my specific case, which involves extracting residuals from a model with inputs and ARMA errors. I tried including the lines:

*ods trace on; ods trace off;
ODS output ResidualNormalityPanel=work.abc;

before my PROC ARIMA code and ending with a quit; after the last run;. Unfortunately, this did not produce the dataset work.abc as expected. Would you be able to clarify if additional modifications are needed to make this approach work for models with inputs and ARMA errors? Thank you once again for your time and assistance!

[ballardw]

---

@sasalex2024 wrote:

Dear Koen,

Thank you very much for your response. Your code works perfectly for the example model you provided. However, I have been unable to adapt it to my specific case, which involves extracting residuals from a model with inputs and ARMA errors. I tried including the lines:

*ods trace on; ods trace off;
ODS output ResidualNormalityPanel=work.abc;

before my PROC ARIMA code and ending with a quit; after the last run;. Unfortunately, this did not produce the dataset work.abc as expected. Would you be able to clarify if additional modifications are needed to make this approach work for models with inputs and ARMA errors? Thank you once again for your time and assistance!

---

Run the trace on/off around your model code and show us the LOG. The log will include a bunch of bits with Names of the ODS objects created, include all of that along with the code and other messages the procedure generates. It may be that your install created a different ODS object than the ResidualNormalityPanel, or the spelling changed, from @sbxkoenk's install.

ODS TRACE ON;
proc arima data=a;
identify var=Yt crosscorr=(X1t X2t) noprint;
estimate q=1 input=( (1) X1t (1) X2t ) method=ml;
run;
quit;

ODS TRACE OFF;

[sasalex2024]

Hi, thank you. This is the log:

1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

NOTE: ODS statements in the SAS Studio environment may disable some output features.

69

70 ODS TRACE ON;

71 proc arima data=series_31;

72 identify var=Yt(1) crosscorr=(X1t(1) X2t(1) X3t(1)) noprint;

73 estimate q=1 input=( (1) X1t (1) X2t (1) X3t) method=ml;

74 run;

NOTE: The ODS graphics associated with the IDENTIFY statement is suppressed because of the presence of NOPRINT option.

Output Added:

-------------

Name: ParameterEstimates

Label: Maximum Likelihood Estimation

Template: ets.Arima.ParameterEstimates

Path: Arima.Estimate.ParameterEstimates

-------------

Output Added:

-------------

Name: FitStatistics

Label: Fit Statistics

Template: ets.Arima.FitStatistics

Path: Arima.Estimate.FitStatistics

-------------

Output Added:

-------------

Name: CorrB

Label: Correlations of Parameter Estimates

Template: ets.Arima.CorrB

Path: Arima.Estimate.CorrB

-------------

Output Added:

-------------

Name: ChiSqAuto

Label: Autocorrelation Check of Residuals

Template: ets.Arima.ChiSqAuto

Path: Arima.Estimate.ChiSqAuto

-------------

Output Added:

-------------

Name: ResidualCorrPanel

Label: Residual Correlation Panel

Template: ets.arima.Graphics.CorrPanel

Path: Arima.Estimate.ResidualCorrPanel

-------------

Output Added:

-------------

Name: ResidualNormalityPanel

Label: Residual Normality Panel

Template: ets.arima.Graphics.NormalityPanel

Path: Arima.Estimate.ResidualNormalityPanel

-------------

Output Added:

-------------

Name: ModelDescription

Label: Model for variable Yt

Template: ets.Arima.ModelDescription

Path: Arima.Estimate.Filter.ModelDescription

-------------

Output Added:

-------------

Name: MAPolynomial

Label: Moving Average Factors

Template: ets.Arima.Equation

Path: Arima.Estimate.Filter.MAPolynomial

-------------

Output Added:

-------------

Name: InputDescription

Label: Input Number 1

Template: ets.Arima.InputDescription

Path: Arima.Estimate.Filter.InputDescription

-------------

Output Added:

-------------

Name: NumPolynomial

Label: Numerator Factors

Template: ets.Arima.Equation

Path: Arima.Estimate.Filter.NumPolynomial

-------------

Output Added:

-------------

Name: InputDescription

Label: Input Number 2

Template: ets.Arima.InputDescription

Path: Arima.Estimate.Filter.InputDescription

-------------

Output Added:

-------------

Name: NumPolynomial

Label: Numerator Factors

Template: ets.Arima.Equation

Path: Arima.Estimate.Filter.NumPolynomial

-------------

Output Added:

-------------

Name: InputDescription

Label: Input Number 3

Template: ets.Arima.InputDescription

Path: Arima.Estimate.Filter.InputDescription

-------------

Output Added:

-------------

Name: NumPolynomial

Label: Numerator Factors

Template: ets.Arima.Equation

Path: Arima.Estimate.Filter.NumPolynomial

-------------

75 quit;

NOTE: PROCEDURE ARIMA used (Total process time):

real time 0.25 seconds

user cpu time 0.13 seconds

system cpu time 0.02 seconds

memory 15596.43k

OS Memory 36812.00k

Timestamp 12/20/2024 09:47:05 PM

Step Count 57 Switch Count 12

Page Faults 0

Page Reclaims 8661

Page Swaps 0

Voluntary Context Switches 531

Involuntary Context Switches 8

Block Input Operations 0

Block Output Operations 952

76

77 ODS TRACE OFF;

78

79 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

89

------------------

This is the code:

ODS TRACE ON;
proc arima data=series_31;
identify var=Yt(1) crosscorr=(X1t(1) X2t(1) X3t(1)) noprint;
estimate q=1 input=( (1) X1t (1) X2t (1) X3t) method=ml;
run;
quit;

ODS TRACE OFF;

[ballardw]

This part of your log:

Output Added:
-------------
Name: ResidualNormalityPanel
Label: Residual Normality Panel
Template: ets.arima.Graphics.NormalityPanel
Path: Arima.Estimate.ResidualNormalityPanel
-------------

Looks like the ods object  you want is the same as @sbxkoenk's code used.

Perhaps you can show the Log where you have the ODS OUTPUT statement with your model.

proc arima data=series_31;
   identify var=Yt(1) crosscorr=(X1t(1) X2t(1) X3t(1)) noprint;
   estimate q=1 input=( (1) X1t (1) X2t (1) X3t) method=ml;
   ODS OUTPUT ResidualNormalityPanel= work.yourdatasetname ;
run;
quit;

I don't have access to Proc Arima so can't run the code. There might be that with procedures supporting run group processing like Proc Arima that the location of the ODS OUTPUT has to be within the run group.

[sasalex2024]

Thank you so much, I have run your code and it works too! It produces the same values as in the method given by Koen. Again, thank you very much.

[sbxkoenk · Accepted solution]

This is a Regression with ARMA Errors

I combine input series with an ARMA model for the error.

The following statements regress SALE on DISCOUNT and PRICE but with the error term of the regression model (called the noise series in ARIMA modeling terminology) assumed to be an ARMA(1,1) process:

ods output ResidualNormalityPanel=work.abc;
proc arima data=sashelp.pricedata;
   where region=1 AND line=1 and product=1;
   identify var=sale crosscorr=(price discount);
   estimate p=1 q=1 input=(price discount);
run;
QUIT;

It works !

Koen

[sasalex2024]

Thank you very much for your help. I ran the following code based on yours (shown below this message), and although the log shows an error: "ERROR: Variable region is not on file WORK.SERIES_31," it still produces the abc table in the Work directory.

I apologize for troubling you again, but there is a column in that table (the 10th column, I believe) named "residual." I assume these are the residuals from the model, correct? And they are sorted I suppose. If so, could you please tell me how I can obtain these residuals in their original, "unsorted" order? I also need to plot them over time.

Thank you again!

----

proc arima data=series_31;
where region=1 AND line=1 and product=1;
identify var=Yt(1) crosscorr=(X1t(1) X2t(1) X3t(1)) noprint;
estimate q=1 input=( (1) X1t (1) X2t (1) X3t) method=ml;

run;
quit;