turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- Analytics
- /
- Stat Procs
- /
- missing values with proc varclus

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

01-31-2012 04:04 PM

Hi,

I'm trying to use Proc Varclus and I'm getting a lot of

ERROR: All observations in data set WORK.PROTEIN have missing values, or the sum of weights or

frequencies is nonpositive.

because of the missing values in my data.

This is my code

proc varclus data=IN outtree=OUT centroid;

var VAR01-VAR200;

run;

Is there a way around it or would proc varclus work fine if I make the missing to 0's. The values are averages of different economical growth rates. So for example average growth rate ot the unemployment rate in the past 5 years, and average GDP growth rate in the past 5 years, etc... Thanks

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

01-31-2012 05:50 PM

For averages of growth rate you're setting 0, which is assuming a stable system. I think 0 would be misleading, especially as every observation is missing a variable.

I'd double check why the variables are missing and seeing if there was something logically incorrect with my calculations or if there was another way to interpolate these numbers than assume 0.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

02-01-2012 08:01 AM

If it is indeed the case that every observation has at least one missing value, you might need to choose another approach. Three ideas:

1) There might be one or two (or 10) variables that contain mostly missing values, whereas the other 999 variables don't contain ANY missings.

If so, you can exclude those "bad" variables and just use the good ones. To count the number of missing values for each variable, use

proc means data=IN NMISS N; var VAR01-Var200; run;

2) If your missing values are scattered at random throughout the variables, you might be able to impute values for the missings by using PROC MI, and then analyze the imputed variables. However, if you have too many missing values (like 60%). I'm not sure I would trust the MI results.

3) Instead of VARCLUS, you can use PROC CORR to look at PAIRWISE correlations between variables. A pairwise approach uses all observations in common between a pair of variables, whereas the LISTWISE approach that you are currently using requires all 200 variables be nonmissinging.There are then various ways to cluster variables based on the pairwise correlations. For example, see

C. Hurley (2004) "Clustering Visualisations of Multidimensional Data", *J. Comp. and Graphical Stats*, 13(4), pp 788-806

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

02-01-2012 09:11 AM

Thanks Rick, I'll try 2 and 3 ...

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

02-01-2012 09:33 AM

Rick, would it also make sense to use PROC EXPAND to intrapolate the missing values (whether they're growth rates or the actual values) and then VARCLUS or PROC SIMILARITY woudl work? Since I do have a Date stamp ..

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

02-01-2012 09:41 AM

It might, if the time series aren't too wild and noisy.

You need to do (1) because if there are variables that contain only a few nonmissing observations, those should be dropped from the analysis. Using PROC EXPAND (or any interpolation/imputation scheme) on data that has only a few valid points is not a good idea.