BookmarkSubscribeRSS Feed
q5pham
Obsidian | Level 7

Hi everyone,

 

Could anyone guide me on steps how to unsmooth a time serie data and which procedures can do the task on SAS? As far as I undestand, I would first need to identify the autocorrelation level and unsmooth in the next step.

 

*1st step: I suppose Proc ARIMA is the one that can estimate smoothing coefficients of my data through the esitmation the orders of AR and MA. I tried to run my tested code:

ods exclude all;

proc arima data=Sample ;
    identify var=performance centre MINIC SCAN  ;
    by code;
    ods output DescStats=DescStat TentativeOrders=orders;
run ;

 

but there is an ERROR: The variance of the time series for variable Performance is smaller than 1E-12. I successfully run in my first trial, but for some reason there were errors in my next trials. My data and variable is unchanged.

*2nd step: To remove the autocorrelation or unsmooth the data - Is there any built-in proc in SAS to do the task?

 

Your help is greatly appreciated as I've been struggling with this for a week now.

 

Thanks, Mai

7 REPLIES 7
q5pham
Obsidian | Level 7

Found the problem. The QUIT statement is missing and it creates a big burden to me.

 

My next question is my data is relatively big, contains more than 5000 groups (the variable code). And SAS can only handle upto some few dozen tables. Is there any way to estimate the theta coefficients of the MA(2) process with my data.

 

Thanks

SteveDenham
Jade | Level 19

Efficient use of ODS OUTPUT and ODS EXCLUDE statements may make what you are asking for much easier.

 

Steve Denham

q5pham
Obsidian | Level 7

Hi Steve,

 

I do use ODS Exclude and Ods Output. My next problem arise. Proc Arima is just a mystery that I cannot figure out what's really happening. I encountered the error: "The variance of the time series for variable Performance is smaller than 1E-12" again. If I run without the BY statement, it perfectly works. Below is my log. Hope you could help:

 

      ods exclude all;
29         ods select InitialMAEstimates;
30         proc arima data=finalsample8 ;
31             by code;
32             identify var=performance center nlag=12 ;
33             estimate q=(1)(1) noconstant ;
34             ods output PrelimEstimates (match_all=q persist=proc)  =MA ;
NOTE: MATCH_ALL=Q is no longer needed if your goal is to create a single data set containing all the variables in all the
      identified objects. By default, if the ODS OUTPUT statement identifies multiple output objects, ODS now combines all the
      objects into a single data set that contains all the variables from all the objects.
35         run ;

NOTE: Interactivity disabled with BY processing.
36         quit;

ERROR: The variance of the time series for variable Performance is smaller than 1E-12.
ERROR: Estimation was not performed because identification was not done.
NOTE: The above message was for the following BY group:
      Code=12406
WARNING: Output 'InitialMAEstimates' was not created.  Make sure that the output object name, label, or path is spelled correctly.  
         Also, verify that the appropriate procedure options are used to produce the requested output object.  For example, verify
         that the NOPRINT option is not used.
NOTE: PROCEDURE ARIMA used (Total process time):
      real time           3:51.11
      cpu time            1:17.81

Rick_SAS
SAS Super FREQ

Sometimes errors like this occur when the BY variable has a level for which there are only one or two values, or for which the data are constant. Use PROC FREQ to examine your BY-group variable:

 

proc freq data=finalsample8 ;

tables code;

run;

 

This will identify the first BY group and tellyou how many observations. If the first BY group has more than a few observations, look at the 'performance variable':

 

proc means data=finalsample8 N NMISS MEAN STD;

WHERE code = /* put value of first level here */;

var performance;

run;

q5pham
Obsidian | Level 7

Hi Rick,

 

My observations in each group are large, min 30 and max 140+.

 

I cannot understand why, but after I change the output table from InitialMAEstimates to MAPolynomial, the error diappears. I suspected it's the error on the table name, instead of the small variance.

 

But it's another mystery. When I ran ods trace on, the InitialMAEstimates did not show up, which means there is no table with that name. But it is listed on https://support.sas.com/documentation/cdl/en/etsug/60372/HTML/default/viewer.htm#etsug_arima_sect052...

 

Now, instead of producing 5000 MAPolynomial tables corresponding to 5000 groups, it only produced 500, the maximum level I can set in the Tools/options/Results/General Results menu.

 

Basically, I would like to estimate the theta coefficients of the MA(2) process. If there are any other suggestion other than Proc Arima, I would really appreciate.

 

Thanks, Mai

 

Rick_SAS
SAS Super FREQ

Comment out the BY statement. Replace it with

WHERE code = /* first level here */;

Debug the code in this simple case.

 

In this simpler situation, I suggest that comment out the ODS SELECT statement and turn on ODS TRACE ON;

q5pham
Obsidian | Level 7
Yeap. I'm testing with one code only for now, using the where code=. It works. But i still have to work out with the remaining 5000 codes

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 7 replies
  • 3208 views
  • 1 like
  • 3 in conversation