Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

Variable reduction

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 13
Accepted Solution

Variable reduction

Hi All,

      I am pretty new to modeling, I am struck with over 900 Interval input variables. I need some ideas to reduce them. Is there any way to find correlation between these variables so that redundancy can be handled? I am using Miner and Guide.


Accepted Solutions
Solution
‎02-16-2015 07:48 AM
SAS Employee
Posts: 340

Re: Variable reduction

Posted in reply to MinalMMurkhande

Hi Minal,

SAS is able to calculate the correlation matrix of those 800 variable. But as you already noted, it is quite hard to look at 800x800/2 coefficients manually.

One way to handle it to calculate the first K principal components (PCA Node in Enterprise Miner), and use them in a predictive model. PCA is based on the correlation matrix. Instead of the original variables you will have K factor scores.

PCA factors and factor scores are hard to interpret, because each factor is a mixture (linear combination) of all variables. With variable clustering you also get factors but each of them depends only on some of the variables. Variable clustering is also based on correlations. It iteratively calculates PCA on the original variables (and on linear combinations of variables).

You can keep 1 variable from each cluster (as you described), or you can keep a linear combination of the variables in the cluster. The former is more interpretable, the latter is more “precise” in some sense.

Gergely

View solution in original post


All Replies
Super Contributor
Posts: 337

Re: Variable reduction

Posted in reply to MinalMMurkhande

Hey Minal,

Enterprise Miner is really easy to learn. Use the reference help a lot (press F1 on your keyboard) and google for white papers on the most common Analytics problems you are trying to solve.

Take a look at this thread where we list a good number of ways to do variable selection: .

Good luck!

Miguel

Occasional Contributor
Posts: 13

Re: Variable reduction

Posted in reply to M_Maldonado

Hi Miguel,

    Thank you for your reply! I had been waiting on someone to answer. But, given a set of census data(400 interval variable)  and financial data(another 400 interval variable) how do I find the correlation within these variables ? Will correlation as the first screening help ? or should I directly start on with decision tree / GBM models?

I can calculate spearman and hoeffding coefficients and also the VIF factor, but all that comes later once I run the model. How do I start of with initial screening? It would be very good if I could screen them using pearson correlation statistic... , but it would give me a matrix with 400 rows and 400 columns Smiley Sad

Also, I did Variable clustering. I select one variable which has the least 1-R^2 in each cluster. Would that work either?

I don't now if I am thinking in the right way. Any help would be appreciated.

Thanks,

Minal

SAS Employee
Posts: 340

Re: Variable reduction

Posted in reply to MinalMMurkhande

Hi,

You didn't mention the purpose of your study. Prediction? What is your target variable?

If census data are predictors, and financial variables are the target, then try PLS.

Yes, Variable Clustering is a good tool for explanatory analysis or for dimensionality reduction.

You can also do a PCA or variable selection (node). Or you can use some of the modeling nodes (tree, forest, regression, LAR/LASSO, PLS, etc.) to select useful variables.
Gergely

Message was edited by: Gergely Bathó

Occasional Contributor
Posts: 13

Re: Variable reduction

Posted in reply to gergely_batho

Hello,

       I have a target variable which is binary. Both census data and finance data are predictors. Each have close to 400 variables so the total number of variables that I need to reduce is 800. Variable Clustering did help. But I was looking for more accurate solutions like correlation etc. Is there no way in which I can find the correlation between these variables? Also, would finding correlation for so many variables work?

Thanks,

Minal

Solution
‎02-16-2015 07:48 AM
SAS Employee
Posts: 340

Re: Variable reduction

Posted in reply to MinalMMurkhande

Hi Minal,

SAS is able to calculate the correlation matrix of those 800 variable. But as you already noted, it is quite hard to look at 800x800/2 coefficients manually.

One way to handle it to calculate the first K principal components (PCA Node in Enterprise Miner), and use them in a predictive model. PCA is based on the correlation matrix. Instead of the original variables you will have K factor scores.

PCA factors and factor scores are hard to interpret, because each factor is a mixture (linear combination) of all variables. With variable clustering you also get factors but each of them depends only on some of the variables. Variable clustering is also based on correlations. It iteratively calculates PCA on the original variables (and on linear combinations of variables).

You can keep 1 variable from each cluster (as you described), or you can keep a linear combination of the variables in the cluster. The former is more interpretable, the latter is more “precise” in some sense.

Gergely

Occasional Contributor
Posts: 13

Re: Variable reduction

Posted in reply to gergely_batho


Thank you Gergely !

Super Contributor
Posts: 337

Re: Variable reduction

Posted in reply to gergely_batho

@Gergely, thanks, that is some solid advice!

@Minal, if you are interested on calculating the VIF, here is one way to approach it: .

Thanks,

Miguel

🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 7 replies
  • 577 views
  • 0 likes
  • 3 in conversation