Hi everyone,
Could anyone guide me on steps how to unsmooth a time serie data and which procedures can do the task on SAS? As far as I undestand, I would first need to identify the autocorrelation level and unsmooth in the next step.
*1st step: I suppose Proc ARIMA is the one that can estimate smoothing coefficients of my data through the esitmation the orders of AR and MA. I tried to run my tested code:
ods exclude all;
proc arima data=Sample ;
identify var=performance centre MINIC SCAN ;
by code;
ods output DescStats=DescStat TentativeOrders=orders;
run ;
but there is an ERROR: The variance of the time series for variable Performance is smaller than 1E-12. I successfully run in my first trial, but for some reason there were errors in my next trials. My data and variable is unchanged.
*2nd step: To remove the autocorrelation or unsmooth the data - Is there any built-in proc in SAS to do the task?
Your help is greatly appreciated as I've been struggling with this for a week now.
Thanks, Mai
Found the problem. The QUIT statement is missing and it creates a big burden to me.
My next question is my data is relatively big, contains more than 5000 groups (the variable code). And SAS can only handle upto some few dozen tables. Is there any way to estimate the theta coefficients of the MA(2) process with my data.
Thanks
Efficient use of ODS OUTPUT and ODS EXCLUDE statements may make what you are asking for much easier.
Steve Denham
Hi Steve,
I do use ODS Exclude and Ods Output. My next problem arise. Proc Arima is just a mystery that I cannot figure out what's really happening. I encountered the error: "The variance of the time series for variable Performance is smaller than 1E-12" again. If I run without the BY statement, it perfectly works. Below is my log. Hope you could help:
ods exclude all;
29 ods select InitialMAEstimates;
30 proc arima data=finalsample8 ;
31 by code;
32 identify var=performance center nlag=12 ;
33 estimate q=(1)(1) noconstant ;
34 ods output PrelimEstimates (match_all=q persist=proc) =MA ;
NOTE: MATCH_ALL=Q is no longer needed if your goal is to create a single data set containing all the variables in all the
identified objects. By default, if the ODS OUTPUT statement identifies multiple output objects, ODS now combines all the
objects into a single data set that contains all the variables from all the objects.
35 run ;
NOTE: Interactivity disabled with BY processing.
36 quit;
ERROR: The variance of the time series for variable Performance is smaller than 1E-12.
ERROR: Estimation was not performed because identification was not done.
NOTE: The above message was for the following BY group:
Code=12406
WARNING: Output 'InitialMAEstimates' was not created. Make sure that the output object name, label, or path is spelled correctly.
Also, verify that the appropriate procedure options are used to produce the requested output object. For example, verify
that the NOPRINT option is not used.
NOTE: PROCEDURE ARIMA used (Total process time):
real time 3:51.11
cpu time 1:17.81
Sometimes errors like this occur when the BY variable has a level for which there are only one or two values, or for which the data are constant. Use PROC FREQ to examine your BY-group variable:
proc freq data=finalsample8 ;
tables code;
run;
This will identify the first BY group and tellyou how many observations. If the first BY group has more than a few observations, look at the 'performance variable':
proc means data=finalsample8 N NMISS MEAN STD;
WHERE code = /* put value of first level here */;
var performance;
run;
Hi Rick,
My observations in each group are large, min 30 and max 140+.
I cannot understand why, but after I change the output table from InitialMAEstimates to MAPolynomial, the error diappears. I suspected it's the error on the table name, instead of the small variance.
But it's another mystery. When I ran ods trace on, the InitialMAEstimates did not show up, which means there is no table with that name. But it is listed on https://support.sas.com/documentation/cdl/en/etsug/60372/HTML/default/viewer.htm#etsug_arima_sect052...
Now, instead of producing 5000 MAPolynomial tables corresponding to 5000 groups, it only produced 500, the maximum level I can set in the Tools/options/Results/General Results menu.
Basically, I would like to estimate the theta coefficients of the MA(2) process. If there are any other suggestion other than Proc Arima, I would really appreciate.
Thanks, Mai
Comment out the BY statement. Replace it with
WHERE code = /* first level here */;
Debug the code in this simple case.
In this simpler situation, I suggest that comment out the ODS SELECT statement and turn on ODS TRACE ON;
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 16. Read more here about why you should contribute and what is in it for you!
ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.
Find more tutorials on the SAS Users YouTube channel.