BookmarkSubscribeRSS Feed
podarum
Quartz | Level 8

Hi,

I'm trying to use Proc Varclus and I'm getting a lot of

ERROR: All observations in data set WORK.PROTEIN have missing values, or the sum of weights or

frequencies is nonpositive.

because of the missing values in my data.

This is my code

proc varclus data=IN outtree=OUT centroid;

var VAR01-VAR200;

run;

Is there a way around it or would proc varclus work fine if I make the missing to 0's. The values are averages of different economical growth rates. So for example average growth rate ot the unemployment rate in the past 5 years, and average GDP growth rate in the past 5 years, etc... Thanks

5 REPLIES 5
Reeza
Super User

For averages of growth rate you're setting 0, which is assuming a stable system. I think 0 would be misleading, especially as every observation is missing a variable.

I'd double check why the variables are missing and seeing if there was something logically incorrect with my calculations or if there was another way to interpolate these numbers than assume 0.

Rick_SAS
SAS Super FREQ

If it is indeed the case that every observation has at least one missing value, you might need to choose another approach. Three ideas:

1) There might be one or two (or 10) variables that contain mostly missing values, whereas the other 999 variables don't contain ANY missings.

If so, you can exclude those "bad" variables and just use the good ones. To count the number of missing values for each variable, use

proc means data=IN NMISS N; var VAR01-Var200; run;

2) If your missing values are scattered at random throughout the variables, you might be able to impute values for the missings by using PROC MI, and then analyze the imputed variables.  However, if you have too many missing values (like 60%). I'm not sure I would trust the MI results.

3) Instead of VARCLUS, you can use PROC CORR to look at PAIRWISE correlations between variables.  A pairwise approach uses all observations in common between a pair of variables, whereas the LISTWISE approach that you are currently using requires all 200 variables be nonmissinging.There are then various ways to cluster variables based on the pairwise correlations. For example, see

C. Hurley (2004) "Clustering Visualisations of Multidimensional Data", J. Comp. and Graphical Stats, 13(4), pp 788-806

podarum
Quartz | Level 8

Thanks Rick, I'll try 2 and 3 ...

podarum
Quartz | Level 8

Rick, would it also make sense to use PROC EXPAND to intrapolate the missing values (whether they're growth rates or the actual values)  and then VARCLUS or PROC SIMILARITY woudl work? Since I do have a Date stamp ..

Rick_SAS
SAS Super FREQ

It might, if the time series aren't too wild and noisy.

You need to do (1) because if there are variables that contain only a few nonmissing observations, those should be dropped from the analysis. Using PROC EXPAND (or any interpolation/imputation scheme) on data that has only a few valid points is not a good idea.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 5 replies
  • 5062 views
  • 3 likes
  • 3 in conversation