topic Re: Principal components for 10000 variable dataset in SAS Programming

Principal components for 10000 variable dataset

gagagu — Tue, 15 Jul 2014 11:18:02 GMT

Hi all,

I have a dataset which i need to run a clustering algorithm on. First step is to reduce the dataset's dimensions as it has 10400 variables. I use PROC PRINCOMP but the resulting files are so huge they wont open in SAS Results Viewer

Does anyone have a suggestion on how i can access the eigenvalues/eigenvectors tables resulting from the PRINCOMP procedure.

Thanks in advance

Re: Principal components for 10000 variable dataset

user24feb — Tue, 15 Jul 2014 11:59:33 GMT

Not sure if I understood your question properly, but if you are simply looking for a way to check where the result data is use "ods trace on" to find out the name (SAS(R) 9.2 Output Delivery System: User's Guide) and "ods output" to get the data.

Re: Principal components for 10000 variable dataset

gagagu — Tue, 15 Jul 2014 12:06:45 GMT

Thank you. I will try your suggestions. Sorry my question wasnt very clear. I think due to the size of the result data, SAS output just fails to open the results which are over 1GB is size.

Re: Principal components for 10000 variable dataset

SteveDenham — Tue, 15 Jul 2014 12:58:54 GMT

Try:

ods listing close;

ods html close;

ods output eigenvalues=eigenvalues eigenvectors=eigenvectors;

proc princomp ....

run;

ods listing;

ods html;

You can dispense with the html stuff if you are doing this in batch.

Steve Denham

Re: Principal components for 10000 variable dataset

Ksharp — Tue, 15 Jul 2014 13:55:52 GMT

or try Cronbach's coefficient Alpha of proc corr , Check the example 5 in its documentation.

or try proc varclus

Re: Principal components for 10000 variable dataset

stat_sas — Tue, 15 Jul 2014 14:11:07 GMT

Do you want to run clustering algorithm on original variables or on principal components?

Re: Principal components for 10000 variable dataset

gagagu — Tue, 15 Jul 2014 15:33:55 GMT

Thanks Steve,

This worked worked. Managed to read the eigenvectors/eigenvalues data from the temp library.

Much appreciated,

Re: Principal components for 10000 variable dataset

gagagu — Tue, 15 Jul 2014 15:40:10 GMT

@ stat@sas ... i want to run clustering on principal components

Just want to say thanks to you all for your responses. I have managed to access the proc princomp results now.

Many thanks guys,