topic Re: Whittle down possible predictor variables in Statistical Procedures

Whittle down possible predictor variables

NKormanik — Tue, 25 May 2021 00:37:00 GMT

1000 possible predictive variables; one target variable -- PROFIT.

Looking for a way to whittle down the possible predictive variables.

Ideally I'd like to end up with one single best. Or a handful.

Last year someone suggested using HPSplit. The results obtained seemed inconclusive. Many, many parameters to guess at in that procedure, and I might have gotten most guesses wrong.

Lately I've been coming across lots of mention of XGBoost -- a new whiz kid on the algorithms block.

Wondering if you all would recommend using that?

Since XGBoost is not natively included in SAS 9.4, however, trying a built-in procedure would be preferred.

Any thoughts greatly appreciated.

Nicholas Kormanik

Re: Whittle down possible predictor variables

Reeza — Tue, 25 May 2021 00:43:44 GMT

You've already explored the "statistical basics" such as Principal Components and Variable Clustering?
Partial Least Squares Regression?

AFAIK XGBoost is a predictive algorithm (and has had good results IME along with LightGBM) not variable selection methodology.

Re: Whittle down possible predictor variables

Ksharp — Tue, 25 May 2021 12:20:59 GMT

As Reeza said, XGBoost or Decision Tree are predict model which are not suited to your data, due to your dependent variable PROFIT is continuous variable ,not category .
Try PROC PLS or PROC GENSELECT :

ods output VariableImportancePlot= VariableImportancePlot;
proc pls data=class missing=em nfac=3 plot=(ParmProfiles VIP) details; * cv=split cvtest(seed=12345);
class sex;
model age=weight height sex;
output out=x predicted=p;
run;

proc hpgenselect data=have ;
class birth_province sex shop_province ;
model profit = ..............

Re: Whittle down possible predictor variables

PaigeMiller — Tue, 25 May 2021 12:45:56 GMT

Strongly recommend PROC PLS, which does not require you to "whittle down" the number of predictor variables. In this paper, the author takes 1000 predictor variables, many of which are highly correlated with another, and creates a useful predictive model without the variable selection step. Please note: the syntax for PROC PLS has changed since that paper was written.

Re: Whittle down possible predictor variables

SteveDenham — Tue, 25 May 2021 12:25:54 GMT

You might want to listen in on @sasmlp 's webinar (see the announcement in this community) where high-dimensional variable selection in SAS will be covered (likely this will emphasize HPGENSELECT).

SteveDenham

Re: Whittle down possible predictor variables

PaigeMiller — Tue, 25 May 2021 13:19:19 GMT

@PaigeMiller wrote:

Strongly recommend PROC PLS, which does not require you to "whittle down" the number of predictor variables. In this paper, the author takes 1000 predictor variables, many of which are highly correlated with another, and creates a useful predictive model without the variable selection step. Please note: the syntax for PROC PLS has changed since that paper was written.

Adding to the above ... PLS is surprisingly robust against multi-collinearity among the X variables, which enables the author to skip the variable selection step. And PLS deserves more attention and more use; although there are probably a thousand published papers now where PLS has been used successfully, it is not widely known amongst data practitioners, and it should be widely known!

Re: Whittle down possible predictor variables

NKormanik — Tue, 25 May 2021 20:14:44 GMT

I greatly appreciate all your suggestions and insights.

Seemingly a straight forward problem. Yet, if the proper tool for the job is not known, one can be stumped forever.

Re: Whittle down possible predictor variables

Ksharp — Wed, 26 May 2021 11:53:35 GMT

So learn statistic/probability theory to know the proper tool for the job .
@Rick_SAS blog is a good place or sas documentation .

Re: Whittle down possible predictor variables

NKormanik — Tue, 01 Jun 2021 05:36:09 GMT

Agree, @Ksharp. @Rick_SAS blogs are very informative. The rub always is, problem at hand is a bit different from the example given. Or, there's an option that who-the-hell knows what to do with it.

If the world were just cookie-cutter clear....