turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- Analytics
- /
- Stat Procs
- /
- multiple linear regression

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 11:18 AM - last edited on 05-08-2017 11:32 AM by Reeza

I use this code to do multiple linear regression:

```
PROC REG DATA=WORK.For_Reg
PLOTS(maxpoints=10000)=ALL
;
Linear_Regression_Model:
MODEL Ln_Amount = ABDOM_HERNIA ADD_PROC ADV_DIABETES BLEED_DISORDERS BR_PR_COL_GI_CANCER
CHF_CARDIOMYO_VALVDIS CHRON_RENAL_FAIL CONVULS CP_MS_OTHER DEP_BIPOLAR_PARA DIAG_PROC DIAG_WO_ADD DIAG_W_ADD
DIG_CONG_ANOM DIVERTICULITIS DRUG_REACT DVT_PE GASTRDUO_ULCER GAST_DUODENITIS GI_BLEED GOUT HEP_CIRR_OTR_LIVER IBS
INTEST_INF INTEST_OBSTR MILD_SLEEP_APNEA MORBID_OBESITY NUTR_OTR_ANEMIA OBESITY RESP_FAILURE RHEU_ARTH SCREEN_WO_ADD
SCREEN_W_ADD SHOCK_SYNC STROKE_PARAL TOBACCO
/ SELECTION=BACKWARD
SLS=0.15
INCLUDE=0
STB CORRB CLB
PCORR1 PCORR2
ALPHA=0.1
COLLIN
;
RUN;
QUIT;
```

I put in 36 independent variables, but SAS dropped 3 vars at the begining, the step 0. Why did sas do that? please see the output.

the output shows:

**The model is not of full rank. A subset of model which is of full rank is choosen.**

Accepted Solutions

Solution

05-08-2017
01:01 PM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 12:58 PM

zhuxiaoyan1 wrote:

I checked. There is only one identical to other one in the model. But I don't know what happened to the other two.

The variables that are causing the problem are identical to *linear combinations* of other variables.

So if a*X5+b*X6+c*X14 = X1 for any real values of a, b and c, then you will get the same error.

As I said, this modeling would be much better handled by Partial Least Squares regression (PROC PLS) where these (and other) problems go away.

All Replies

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 11:30 AM

How many observations do you have?

Are all your variables continuous or did you create any indicator variables?

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 11:55 AM

I have 8048 obs. My variables are indicator variables, not continuous, but my dependent variables are continuous variables. I take logarithm on the Amount. You can see the name Ln_Amount. This is actually a Poisson regression.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 12:00 PM

ln_amount could easily be loan amount, we can't make inferences based on variable names.

If your variables are indicator variables did you remember to exclude one for each categorical variable?

ie for a variable with 6 categories you only have 5 variables?

To determine which variable is the issue you can run a proc freq between them to do the comparisons.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 12:19 PM

All my independent variables are flag as 0 or 1. The Amount is Allowed Amount. My data is health insurance claim data. Insurance company has contract with providers how much they can charge for a particular procedure.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 12:31 PM

I checked. There is only one identical to other one in the model. But I don't know what happened to the other two.

Solution

05-08-2017
01:01 PM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 12:58 PM

zhuxiaoyan1 wrote:

I checked. There is only one identical to other one in the model. But I don't know what happened to the other two.

The variables that are causing the problem are identical to *linear combinations* of other variables.

So if a*X5+b*X6+c*X14 = X1 for any real values of a, b and c, then you will get the same error.

As I said, this modeling would be much better handled by Partial Least Squares regression (PROC PLS) where these (and other) problems go away.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 02:20 PM

Thank you very much for your help! Is there a way to know a variable is identical to linear combinations of other variables? Thanks again!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 02:30 PM

PROC FREQ and you'll have only entries along a diagonal.

Knowing your data - sometimes the same variable is defined multiple ways and included or procedures always go together so including both doesn't make sense.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 02:45 PM - edited 05-08-2017 02:54 PM

Reeza wrote:

PROC FREQ and you'll have only entries along a diagonal.

Knowing your data - sometimes the same variable is defined multiple ways and included or procedures always go together so including both doesn't make sense.

This might find cases where X1=X2, but it won't find cases where a*X1+b*X2+c*X3 = d*X4+e*X5.

PROC PRINCOMP should work, in the above example it will find cases where a*X1+b*X2+c*X3–d*X4+e*X5 has zero variability (or in PROC PRINCOMP language, the component has a zero eigenvalue). And as I have said repeatedly, the use of Partial Least Squares (PROC PLS) avoids this complication entirely.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-09-2017 08:23 AM

Adding to my comment above

And as I have said repeatedly, the use of Partial Least Squares (PROC PLS) avoids this complication entirely.

And as I have said repeatedly, the use of Partial Least Squares (PROC PLS) avoids this complication entirely, and has many other benefits when you are trying to model with 36 independent variables.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-09-2017 09:57 AM

I'll try proc pls. Thank you very much!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 11:31 AM

This is based upon the mathematics of linear regression.

When the message says that the model is not full rank, this means that three of your variables are identical to linear combinations of the other variables, and so cannot be estimated.

You need to determine why your variables are indentical to linear combinations of the other variables, and then delete the variables to eliminate this identicality.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 11:42 AM

I also add my usual advice against Stepwise regression, and that when you have 36 independent variables, you'd be much better off performing Partial Least Squares regression (PROC PLS), rather than stepwise or ordinary least squares regression.

If you search for "problems with stepwise regression" you will find lots of authors writing on this topic. PLS has been shown to provide better estimates of slopes and predicted values than OLS/stepwise (better meaning lower mean squared error), see http://amstat.tandfonline.com/doi/abs/10.1080/00401706.1993.10485033

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-08-2017 12:43 PM

You have two questions.

1. You get the error because you have categorical variables that end up being identical

2. Regarding why it dropped three right of the bat - missing data in those variables, or the p-value is less than 0.15 since you set SLS to 0.15. It should produce a table that shows the reason at each step.