turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- SAS Programming
- /
- SAS Procedures
- /
- Using Proc Reg with categorical variables

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-17-2009 02:56 PM

My goal is to develop a model with proc reg using both categorical (such as gender) and and continuous variables (such as age) to predict a continuous outcome (such as profit). Can proc reg do this? If not what can? If it can, do I specify which ones are categorical? Lastly, I also have a zipcode predictor which can take on 10 values, how do I change this into a binary predictor for regression?

Sorry if these questions are obvious, I'm still learning how to navigate SAS documentation.

-Thanks

Sorry if these questions are obvious, I'm still learning how to navigate SAS documentation.

-Thanks

Accepted Solutions

Solution

03-28-2017
08:00 AM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-20-2009 08:53 AM - last edited on 03-28-2017 07:59 AM by ChrisHemedinger

PROC REG does not support categorical predictors directly. You have to recode them into a series of 0-1 values and use them in the model. A two-level categorical variable (like gender) becomes a simple 0-1 recode and then treated as continuous. A three-level categorical variable becomes two variables, etc.

This is analogous to the reference cell recoding that can be used in PROC GLM for categorical variables. The place that it falls down is that if you use the variable selection tools in REG, then you can end up with the situation of part of a variable in the model.

@Paige agrees:

I would use PROC GLM instead of PROC REG. Your predictor variables that are categories (gender, zip) are placed in the CLASS statement.

Also consider GLMSELECT procedure. It fills the gap of allowing variable selection with CLASS variables. It also produces output that allow further analyses with REG and/or GLM. GLMSELECT treats a class variable as a single multi-degree of freedom test for inclusion/exclusion. Either all levels are in or all levels are out; it's not a piecemeal process.

@Paige points out:

One thing that I feel needs to be pointed out here is that, despite the introduction of PROC GLMSELECT by SAS, many statisticians feel that STEPWISE (including forward and backward) model selection procedures is dangerous and misleading, and advise against using such. (Yes, I know there are other selection procedures in GLMSELECT, such as LAR and LASSO, which I have no knowledge of)

*Editor's note: this response consolidates several of the helpful replies in this thread. Read through the entire topic to see the conversation.*

All Replies

Solution

03-28-2017
08:00 AM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-20-2009 08:53 AM - last edited on 03-28-2017 07:59 AM by ChrisHemedinger

PROC REG does not support categorical predictors directly. You have to recode them into a series of 0-1 values and use them in the model. A two-level categorical variable (like gender) becomes a simple 0-1 recode and then treated as continuous. A three-level categorical variable becomes two variables, etc.

This is analogous to the reference cell recoding that can be used in PROC GLM for categorical variables. The place that it falls down is that if you use the variable selection tools in REG, then you can end up with the situation of part of a variable in the model.

@Paige agrees:

I would use PROC GLM instead of PROC REG. Your predictor variables that are categories (gender, zip) are placed in the CLASS statement.

Also consider GLMSELECT procedure. It fills the gap of allowing variable selection with CLASS variables. It also produces output that allow further analyses with REG and/or GLM. GLMSELECT treats a class variable as a single multi-degree of freedom test for inclusion/exclusion. Either all levels are in or all levels are out; it's not a piecemeal process.

@Paige points out:

One thing that I feel needs to be pointed out here is that, despite the introduction of PROC GLMSELECT by SAS, many statisticians feel that STEPWISE (including forward and backward) model selection procedures is dangerous and misleading, and advise against using such. (Yes, I know there are other selection procedures in GLMSELECT, such as LAR and LASSO, which I have no knowledge of)

*Editor's note: this response consolidates several of the helpful replies in this thread. Read through the entire topic to see the conversation.*

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-20-2009 09:52 AM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-21-2009 11:29 AM

Thanks for the help.

I ran the regression with both PROC REG (created dummy variables) and PROC GLM. For the 10 values of the discrete variable, I created 9 dummy variables.

Also I noticed using proc reg that out of my 9 categorical variables coefficients, that one of them wasn't significant so I dropped it out of my model, does PROC GLM do this?

Lastly,

The Proc Glm doesn't give me estimation parameters, what is the synatx to get that?

I ran the regression with both PROC REG (created dummy variables) and PROC GLM. For the 10 values of the discrete variable, I created 9 dummy variables.

Also I noticed using proc reg that out of my 9 categorical variables coefficients, that one of them wasn't significant so I dropped it out of my model, does PROC GLM do this?

Lastly,

The Proc Glm doesn't give me estimation parameters, what is the synatx to get that?

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-21-2009 01:05 PM

> Thanks for the help.

>

> I ran the regression with both PROC REG (created

> dummy variables) and PROC GLM. For the 10 values of

> the discrete variable, I created 9 dummy variables.

>

> Also I noticed using proc reg that out of my 9

> categorical variables coefficients, that one of them

> wasn't significant so I dropped it out of my model,

> does PROC GLM do this?

This is a problem, as it doesn't make sense to delete the coefficient of one of the 9 categorical variables that represent a single discrete variable. I would recommend that you don't do this.

Since you said that you yourself dropped the term out of the model ... yes, PROC GLM does this as well.

> Lastly,

> The Proc Glm doesn't give me estimation parameters,

> what is the synatx to get that?

In the MODEL statement, use the SOLUTION option.

>

> I ran the regression with both PROC REG (created

> dummy variables) and PROC GLM. For the 10 values of

> the discrete variable, I created 9 dummy variables.

>

> Also I noticed using proc reg that out of my 9

> categorical variables coefficients, that one of them

> wasn't significant so I dropped it out of my model,

> does PROC GLM do this?

This is a problem, as it doesn't make sense to delete the coefficient of one of the 9 categorical variables that represent a single discrete variable. I would recommend that you don't do this.

Since you said that you yourself dropped the term out of the model ... yes, PROC GLM does this as well.

> Lastly,

> The Proc Glm doesn't give me estimation parameters,

> what is the synatx to get that?

In the MODEL statement, use the SOLUTION option.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-21-2009 01:51 PM

Paige, thank you for all your help.

Last question, should I be alarmed when I get the following note:

NOTE: The X'X matrix has been found to be singular, and a generalized inverse was used to solve the

normal equations. Terms whose estimates are followed by the letter 'B' are not uniquely

estimable.

Last question, should I be alarmed when I get the following note:

NOTE: The X'X matrix has been found to be singular, and a generalized inverse was used to solve the

normal equations. Terms whose estimates are followed by the letter 'B' are not uniquely

estimable.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-21-2009 02:33 PM

The X'X message is generally benign if you have a CLASS statement in GLM. It is used to show all levels of the CLASS variable in the analysis. Do be concerned if there are TWO rows labeled "B" for one variable, as that represents and unplanned linear dependency.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-21-2009 02:47 PM

I've got 9 rows with the B in it.

'Do be concerned if there are TWO rows labeled "B" for one variable, as that represents and unplanned linear dependency. '

Are you saying I should only be worried if there are two rows labeled 'B' and I don't have a class statement? Message was edited by: Jrb599

'Do be concerned if there are TWO rows labeled "B" for one variable, as that represents and unplanned linear dependency. '

Are you saying I should only be worried if there are two rows labeled 'B' and I don't have a class statement? Message was edited by: Jrb599

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-21-2009 04:28 PM

It is not a concern if the rows are all of the dummy variables (in PROC REG) or levels (in PROC GLM) representing a discrete variable. It is, in fact, an expected outcome when dummy variables (in PROC REG) or discrete variables (in PROC GLM) are used.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-25-2009 01:28 PM

After talking to my professor, I have decided I am going to drop just one of the dummy variables; how do I go about doing this in PROC GLM. Thanks.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-27-2009 08:55 AM

You can't drop just one dummy variable in PROC GLM.

The choice of dummy variables is done internally, so you have no control over it.

Furthermore, the results you get from the PROC GLM way of doing things produces the exact same predictions, exact same sum of squares, exact same model, etc. as any other way of creating dummy variables.

In other words, if you could drop one of the dummy variables, so instead of*n* dummy variables representing *n* levels, you now have *n*–1 dummy variables, the results are the same.

I am surprised your professor does not know this.

The choice of dummy variables is done internally, so you have no control over it.

Furthermore, the results you get from the PROC GLM way of doing things produces the exact same predictions, exact same sum of squares, exact same model, etc. as any other way of creating dummy variables.

In other words, if you could drop one of the dummy variables, so instead of

I am surprised your professor does not know this.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-27-2009 09:08 AM

We do get it, it's the fact that Cat9 and Cat10 have no significant difference and therefore there is no need for that term with such a high p-value. They both can be estimated by the parameter without developing a poor model.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-27-2009 12:37 PM

Hi Jrb599,

A point to remember. The dummy variable that is not in the model represents a reference level for the categorical variable represented by the dummy variables in the model. This was mentioned by Doc@Duke at the beginning of this thread. The reference level is the one to which all other levels are compared. Therefore, if more than one dummy variables are not in the model, then all other levels are compared to the combination of the levels these dummy variables represent.

A point to remember. The dummy variable that is not in the model represents a reference level for the categorical variable represented by the dummy variables in the model. This was mentioned by Doc@Duke at the beginning of this thread. The reference level is the one to which all other levels are compared. Therefore, if more than one dummy variables are not in the model, then all other levels are compared to the combination of the levels these dummy variables represent.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-27-2009 01:02 PM

"Hi Jrb599,

A point to remember. The dummy variable that is not in the model represents a reference level for the categorical variable represented by the dummy variables in the model. This was mentioned by Doc@Duce at the beginning of this thread. The reference level is the one to which all other levels are compared. Therefore, if more than one dummy variables are not in the model, then all other levels are compared to the combination of the levels these dummy variables represent."

Yes I get this, this is what I'm trying to do since there is no difference between Cat9 and Cat10(my reference level).

A point to remember. The dummy variable that is not in the model represents a reference level for the categorical variable represented by the dummy variables in the model. This was mentioned by Doc@Duce at the beginning of this thread. The reference level is the one to which all other levels are compared. Therefore, if more than one dummy variables are not in the model, then all other levels are compared to the combination of the levels these dummy variables represent."

Yes I get this, this is what I'm trying to do since there is no difference between Cat9 and Cat10(my reference level).

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-27-2009 12:44 PM

Jrb599,

One thing that I had forgotten, as it is so new to SAS, is the SAS 9.2 procedure GLMSELECT. It fills the gap of allowing variable selection with CLASS variables. It also produces output that allow further analyses with REG and/or GLM.

I haven't tried it, but it may help address some of the questions that you have posed here.

Doc Muhlbaier

Duke

One thing that I had forgotten, as it is so new to SAS, is the SAS 9.2 procedure GLMSELECT. It fills the gap of allowing variable selection with CLASS variables. It also produces output that allow further analyses with REG and/or GLM.

I haven't tried it, but it may help address some of the questions that you have posed here.

Doc Muhlbaier

Duke