Unobserved Components Model: Negetive R square and missing Adjusted R square

[jaweriahh]

I want to estimate the unobserved components model with annual time series data for 55 years. Since my data is in annual term I leave out the seasonal component and fit a trend-cycle model. Moreover I include dummy variable for structural breaks and outliers in the data.

I check whether all the components are stochastic and contributing to the model and set the non stochastic terms to zero using the level variance=0 and slope variance=0.

However when I estimate my final model the R-Square of my model is negative and the Adjusted R-Square is missing. how do i get proper estimates for the R-Square and the Adjusted R-Square?

Thank you

[udo_sas]

Hello -

This is the proper estimate, negative R-square typically indicates a poor fit of your model to your data.

http://support.sas.com/documentation/cdl/en/etsug/68148/HTML/default/viewer.htm#etsug_ucm_details40.... of the PROC UCM documentation describes how the R-square and other statistics of fit are computed by the procedure. As you can see the R-square statistic is defined in such a way that it compares the fitted model (SSE) to a simple mean model (SST). If the fitted model does not fit the data as well as a simple mean model, then the R-square statistic will be negative.

Having said all of this, my suggestion is to not use R-square for assessing time series models. A good explanation can be found here: http://stats.stackexchange.com/questions/101546/what-is-the-problem-with-using-r-squared-in-time-ser...

Thanks,

Udo

[jaweriahh]

Thank for for your reply and the links. My data shows a linear trend with possible cyclical patterns. I use the code

proc ucm data = metals;
id year interval = year;
model cop;
irregular;
level;
slope;
cycle;
cycle;
estimate;
run;

 and modify it to make the slope deterministic and include structural breaks and outlier dummies.

proc ucm data = metals;
id year interval = year;
model cop=break1987 break2005 break1977 outlier2008 outlier1979 outlier2003 outlier1993 outlier1975;
irregular;
level;
slope variance=0 noest;
cycle;
cycle;
estimate;
run;

the model confirms two cycles (one deterministic and one stochastic), stochastic level and fixed slope. The model conforms to normality and the model selection criteria are better than the initial model.  

I am attaching the results. given the diagonostic tests and results is a low R square a concern in this model?

[udo_sas]

Hello -

Don't use R-Square for assessing time series models.

If you want to assess your model in a proper way you should consider using out-of-sample data (assuming we are talking about a forecasting exercise).

Would it be possible for you to share your data?

Thanks,

Udo

[jaweriahh]

Thanks for your reply.

I have two objectives. The first is to decompose the data and analyze the individual components the second is to forecast. I have two date series and mentioned the code I am using.

proc ucm data = metals;
id year interval = year;
model al=outlier1979 outlier2008 break1973 outlier1994;
irregular;
level;
slope;
cycle;
cycle;
estimate;
run;

proc ucm data = metals;
id year interval = year;
model al=outlier1979 outlier2008 break1973 outlier1994;
irregular;
level variance=0 noest;
slope variance=0 noest;
cycle;
cycle;
estimate;
run;

and 

proc print data=metals;
run;
proc ucm data = metals;
id year interval = year;
model zinc=break1973 outlier2008 outlier2006 outlier1995;
irregular;
level;
slope;
cycle;
cycle;
estimate;
run;

proc ucm data = metals;
id year interval = year;
model zinc=break1973 outlier2008 outlier2006 outlier1995;
irregular;
level;
slope variance=0 noest;
cycle;
cycle;
estimate;
run;

proc ucm data = metals;
id year interval = year;
model zinc=break1973 outlier2008 outlier2006 outlier1995;
irregular;
level variance=0 noest;
slope variance=0 noest;
cycle;
cycle;
estimate;
run;