Hello.
I'm using the ATSM package of the Proc TSMODEL. I’d like to know if the EXTREME string of the PREFILTER option of the DIAGSPEC.SetOption Method, is equivalent to the OUTLIER OPTION of the ARIMAX statement of the PROC HPF DIAGNOSE. Here below the 2 statements (the first for the PROC TSMODEL and the second for the PROC HPFDIAGNOSE).
Many thanks in advance
Someone else might give a better response than this but in the meantime, here is what I would say:
The diagspec object has a dedicated setARIMAXOutlier method that is closer to the OUTLIER option in the ARIMAX statement of HPFDIAGNOSE procedure. The EXTREME string of the PREFILTER option is associated with a quick way to preprocess the series for diagnosis. It should be the same as the PREFILTER=EXTREME option in the HPFDIAGNOSE procedure (in its PROC statement).
Someone else might give a better response than this but in the meantime, here is what I would say:
The diagspec object has a dedicated setARIMAXOutlier method that is closer to the OUTLIER option in the ARIMAX statement of HPFDIAGNOSE procedure. The EXTREME string of the PREFILTER option is associated with a quick way to preprocess the series for diagnosis. It should be the same as the PREFILTER=EXTREME option in the HPFDIAGNOSE procedure (in its PROC statement).
I am glad and you are very welcome.
%let sel_in0 =lscas.level_rp;
%let sel_by =region prefecture;
%let sel_suf =rp_dl;
%let sel_target=avg_download_kbps;
%let fore_x =num_of_cells2G3G added_cells2G3G num_of_cells4G added_cells4G num_of_cells_tot added_cells_tot
weekendgg holidaygg pontegg festivi gg lavorativi;
%let end_fit ="01jul2018"d;
%let sel_back =6;
%let sel_lead =18;
%let sel_in=&sel_in0.;
data _null_;
array conta (*) &fore_x.;
call symput('nvar', compress(dim(conta)));
vars="&fore_x.";
do i=1 to dim(conta);
var=scan(vars,i," ");
call symput('var' || compress(i),compress(var));
end;
run;
%macro atsm;
proc tsmodel
data=&sel_in.
outobj= (
outfor = LSCAS.&sel_suf._LS_OUTFOR /* actual and forecast value*/
outstat = LSCAS.&sel_suf._LS_OUTSTAT /* forecast stat and perfomance */
outmodelinfo = LSCAS.&sel_suf._LS_MODELINFO /* selected model summary */
parmest = LSCAS.&sel_suf._LS_PARMEST /* selected model details */
)
seasonality=12;
*define time series ID variable and the time interval;
id date interval = MONTH FORCEINPUTFORMAT SETMISSING=missing;
*define time series and the corresponding accumulation methods;
var &sel_target. / accumulate=average;
%do v=1 %to &nvar.;
var &&var&v.. / accumulate=average;
%end;
*define the by variables if exist;
by &sel_by.;
*using the ATSM (Automatic Time Series Model) package;
require atsm;
*starting user script;
submit;
*DECLARING THE ATSM OBJECTS;
declare object dataframe(tsdf) ; /* TSDF: Time series data frame used to group series variables for DIAGNOSE and FORENG objects */
declare object diagspec(diagspec); /* DIAGSPEC: Diagnostic control options for DIAGNOSE object */
declare object diagnose(diagnose); /* DIAGNOSE: Automatic time series model generation */
declare object forecast(foreng) ; /* FORENG: Automatic time series model selection and forecasting */
declare object outfor(outfor); /* OUTFOR : Collector for FORENG forecasts */
declare object outstat(outstat); /* OUTSTAT : Collector for FORENG forecast performance statistics */
declare object outmodelinfo(outmodelinfo); /* OUTMODEL: Collector for selected model summary */
declare object parmest(outest); /* PARMEST : Collector for selected model details */
array perror[2] / nosymbols; /* MINIC (ARMA selection) parameters array */
array p[2] / nosymbols; /* AR components parameters array */
array q[2] / nosymbols; /* MA components parameters array */
array ps[2] / nosymbols; /* Seasonal AR components parameters array */
array qs[2] / nosymbols; /* Seasonal MA components parameters array */
array xnum[2] / nosymbols; /* Transfer Function AR components parameters array */
array xden[2] / nosymbols; /* Transfer Function MA components parameters array*/
array diff[2] / nosymbols; /* integration test*/
array shift[2] / nosymbols; /* level shift min duration*/
*INITIALIZE THE TSDF OBJECT AND ASSIGN THE TIME SERIES ROLES;
rc = dataframe.initialize();
/* Target series */
rc = dataframe.addY(&sel_target.);
/* Explantory series */
%do v=1 %to &nvar.;
rc = dataframe.addX(&&var&v.., 'extend','LAST','required','MAYBE','keep',0,'nodiff',0,'control',1,'repmiss',1);
%end;
*OPEN THE DIAGSPEC OBJECT AND ENABLE ESM AND ARIMAX MODEL CLASS FOR DIAGNOSE;
rc = diagspec.open();
rc = diagspec.setOption(
'SIGLEVEL', 0.05 /* Significance level */
,'CRITERION','GMAPE' /* model selection criterion to select the best mode */
,'INPUTMISSPCT',1 /* accetable size (%)of the missing observation of the input time se-ries*/
,'ENTRYPCT', 0.1 /* min percentage increment of the criterion between two candidate models to stop */
,'PREFILTER','EXTREME' /* smoothed values for missing and extreme values prior to diagnostic tests*/
,'TESTINPUT','BOTH' /* log transform and trend testing of the input variables */
,'SELECTINPUT','SELECT' /* maximum number of the input variables to select */
);
PERROR[1]=0;PERROR[2]=24; /* AR order range for MINIC (ARMA selection) error */
P[1] =0;P[2] =12; /* Nonseasonal AR order range */
Q[1] =0;Q[2] =12; /* Nonseasonal MA order range */
PS[1] =0;PS[2] =2; /* Seasonal AR order range */
QS[1] =0;QS[2] =2; /* Seasonal MA order range */
XNUM[1] =0;XNUM[2] =12; /* Transfer Funtcion Numerator order range */
XDEN[1] =0;XDEN[2] =12; /* Transfer Funtcion Denominator order range */
DIFF[1] =0;DIFF[2] =12; /* integration test */
SHIFT[1] =2;SHIFT[2]=6 ; /* integration test */
rc=diagspec.setTrend('SIGLEVEL', 0.05, 'DIFFRAN',DIFF, 'SDIFF','AUTO');
rc = diagspec.setARIMAX(
'SIGLEVEL', 0.2 /* ARIMAX parameters significance*/
,'IDENTIFY','BOTH' /* Identification order (ARIMA, REG, or BOTH) */
,'NOINT',0 /* Suppress constant term (0 | 1) */
,'PERROR',PERROR /* AR order range for MINIC (tentative ARMA order selection) error */
,'P',P /* Nonseasonal AR order range */
,'Q',Q /* Nonseasonal MA order range */
,'PS',PS /* Seasonal AR order range */
,'QS',QS /* Seasonal MA order range */
,'XNUM',XNUM /* Transfer Funtcion Numerator order range */
,'XDEN',XDEN /* Transfer Funtcion Denominator order range */
,'CRITERION','SBC' /* Identification criterion */
,'ESTMETHOD','CLS' /* ARIMA estimation method */
,'METHOD','MINIC' /* Tentative ARMA orders */
);
/* outliers detections */
rc=diagspec.setARIMAXOutlier(
'ALLOWAO',1
,'ALLOWLS',1
,'ALLOWTLS',1
,'DETECT','MAYBE'
,'ENTRYPCT',0.1
,'FILTER','FULL'
,'MAXPCT',5
,'SIGLEVEL',0.01
,'TLSVALS',SHIFT);
/* refine parameters of selected model excluding outliers */
rc = diagspec.setARIMAXRefine(
'SIGLEVEL',0.4
,'ORDER','ARMA:INPUT');
/* UCM */
rc = diagspec.setUCM( 'SIGLEVEL', 0.05
,'IRREGULAR',1
,'LEVEL',1
,'SLOPE',1
,'SEASON',1
,'CYCLE',1
,'AUTOREG',1
,'DEPLAG',1
);
/* refine parameters of selected model excluding outliers */
rc=diagspec.setUCMRefine('SIGLEVEL',0.4,'ORDER','INPUT');
/* cominbed model specification */
rc=diagspec.setCombine();
rc = diagspec.Close();
*SET THE DIAGNOSE OBJECT USING THE DIAGSPEC OBJECT AND RUN THE DIAGNOSE PROCESS;
rc = diagnose.initialize(dataframe);
rc = diagnose.setSpec(diagspec); /* use diagspec defined above to run diagnose */
rc = diagnose.SetOption('HOLDOUT', &sel_back.); /* number of periods to be used only for model selection, not identification */
rc = diagnose.SetOption('BACK' , &sel_back.); /* number of periods to be used for forecast (as if actuals does not exist) */
rc = diagnose.run();
*INITIALIZE THE FORENG OBJECT WITH THE DIAGNOSE RESULT AND RUN MODEL SELECTING AND GENERATE FORECASTS;
rc = forecast.initialize(diagnose); /* use diagnose result for forecasting */
rc = forecast.setOption('ALPHA', 0.05); /* fit statistics*/
rc = forecast.setOption('CRITERION','GMAPE'); /* fit statistics*/
rc = forecast.SetOption('HOLDOUT', &sel_back.); /* number of periods to be used only for model selection, not identification */
rc = forecast.SetOption('BACK', &sel_back.); /* number of periods to be used for forecast (as if actuals does not exist) */
rc = forecast.setOption('LEAD', &sel_lead.); /* number of periods (from back) for which has to be calculate forecast */
rc = forecast.setOption('MINOBS.TREND',12); /*no trend model is fitted to any series with fewer nonmissing obs */
rc = forecast.setOption('MINOBS.SEASON',24); /*no season model is fitted to any series with fewer nonmissing obs */
rc= forecast.setOption('TASK','FORECAST'); /* SELECT=model selection, estimates parameters,produces forecasts
FIT=estimates parameters by using the model specified then forecasts
UPDATE=estimates parameters by using the model specified then forecasts.
UPDATE differs from FIT, parameters found in specified are used
as starting values in estimation
FORECAST=forecasts using model and parameters specified*/
rc = forecast.run();
*COLLECT THE FORECAST AND STATISTIC-OF-FIT FROM THE FORGEN OBJECT RUN RESULTS;
rc = outfor.collect(forecast); /* OUTFOR : load forecasts */
rc = outstat.collect(forecast); /* OUTSTAT : load forecast performance statistics */
rc = outmodelinfo.collect(forecast); /* OUTMODEL: load selected model summary */
rc = parmest.collect(forecast); /* PARMEST : load selected model details */
endsubmit;
quit;
%mend atsm;
option mprint;
%atsm;
This new issue is something that will likely need to be investigated by Tech Support. Please open a Tech Support track using the information in the link below:
https://support.sas.com/en/technical-support/contact-sas.html
In addition to the code included in your recent posting, please also provide the following information to Tech Support:
1) Sample data which reproduces the behavior you are observing,
2) Details about your CAS session setup, such as the number of nodes, cores per node, etc. (Note: This information can be found in the listing file generated by PROC TSMODEL.)
Thank you!
DW
SAS Innovate 2025 is scheduled for May 6-9 in Orlando, FL. Sign up to be first to learn about the agenda and registration!
Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.
Find more tutorials on the SAS Users YouTube channel.