Help using Base SAS procedures

Using Proc Reg with categorical variables

Accepted Solution Solved
Reply
N/A
Posts: 0
Accepted Solution

Using Proc Reg with categorical variables

My goal is to develop a model with proc reg using both categorical (such as gender) and and continuous variables (such as age) to predict a continuous outcome (such as profit). Can proc reg do this? If not what can? If it can, do I specify which ones are categorical? Lastly, I also have a zipcode predictor which can take on 10 values, how do I change this into a binary predictor for regression?

Sorry if these questions are obvious, I'm still learning how to navigate SAS documentation.

-Thanks

Accepted Solutions
Solution
‎03-28-2017 08:00 AM
Trusted Advisor
Posts: 2,115

Re: Using Proc Reg with categorical variables

[ Edited ]
Posted in reply to deleted_user

PROC REG does not support categorical predictors directly. You have to recode them into a series of 0-1 values and use them in the model. A two-level categorical variable (like gender) becomes a simple 0-1 recode and then treated as continuous. A three-level categorical variable becomes two variables, etc.

This is analogous to the reference cell recoding that can be used in PROC GLM for categorical variables. The place that it falls down is that if you use the variable selection tools in REG, then you can end up with the situation of part of a variable in the model.

 

@Paige agrees:

I would use PROC GLM instead of PROC REG. Your predictor variables that are categories (gender, zip) are placed in the CLASS statement.

Also consider GLMSELECT procedure.   It fills the gap of allowing variable selection with CLASS variables. It also produces output that allow further analyses with REG and/or GLM.  GLMSELECT treats a class variable as a single multi-degree of freedom test for inclusion/exclusion. Either all levels are in or all levels are out; it's not a piecemeal process.

 

@Paige points out:

One thing that I feel needs to be pointed out here is that, despite the introduction of PROC GLMSELECT by SAS, many statisticians feel that STEPWISE (including forward and backward) model selection procedures is dangerous and misleading, and advise against using such. (Yes, I know there are other selection procedures in GLMSELECT, such as LAR and LASSO, which I have no knowledge of)

Editor's note: this response consolidates several of the helpful replies in this thread.  Read through the entire topic to see the conversation.

View solution in original post


All Replies
Solution
‎03-28-2017 08:00 AM
Trusted Advisor
Posts: 2,115

Re: Using Proc Reg with categorical variables

[ Edited ]
Posted in reply to deleted_user

PROC REG does not support categorical predictors directly. You have to recode them into a series of 0-1 values and use them in the model. A two-level categorical variable (like gender) becomes a simple 0-1 recode and then treated as continuous. A three-level categorical variable becomes two variables, etc.

This is analogous to the reference cell recoding that can be used in PROC GLM for categorical variables. The place that it falls down is that if you use the variable selection tools in REG, then you can end up with the situation of part of a variable in the model.

 

@Paige agrees:

I would use PROC GLM instead of PROC REG. Your predictor variables that are categories (gender, zip) are placed in the CLASS statement.

Also consider GLMSELECT procedure.   It fills the gap of allowing variable selection with CLASS variables. It also produces output that allow further analyses with REG and/or GLM.  GLMSELECT treats a class variable as a single multi-degree of freedom test for inclusion/exclusion. Either all levels are in or all levels are out; it's not a piecemeal process.

 

@Paige points out:

One thing that I feel needs to be pointed out here is that, despite the introduction of PROC GLMSELECT by SAS, many statisticians feel that STEPWISE (including forward and backward) model selection procedures is dangerous and misleading, and advise against using such. (Yes, I know there are other selection procedures in GLMSELECT, such as LAR and LASSO, which I have no knowledge of)

Editor's note: this response consolidates several of the helpful replies in this thread.  Read through the entire topic to see the conversation.

Super Contributor
Posts: 281

Re: Using Proc Reg with categorical variables

Posted in reply to deleted_user
I would use PROC GLM instead of PROC REG. Your predictor variables that are categories (gender, zip) are placed in the CLASS statement.
N/A
Posts: 0

Re: Using Proc Reg with categorical variables

Posted in reply to deleted_user
Thanks for the help.

I ran the regression with both PROC REG (created dummy variables) and PROC GLM. For the 10 values of the discrete variable, I created 9 dummy variables.

Also I noticed using proc reg that out of my 9 categorical variables coefficients, that one of them wasn't significant so I dropped it out of my model, does PROC GLM do this?

Lastly,
The Proc Glm doesn't give me estimation parameters, what is the synatx to get that?
Super Contributor
Posts: 281

Re: Using Proc Reg with categorical variables

Posted in reply to deleted_user
> Thanks for the help.
>
> I ran the regression with both PROC REG (created
> dummy variables) and PROC GLM. For the 10 values of
> the discrete variable, I created 9 dummy variables.
>
> Also I noticed using proc reg that out of my 9
> categorical variables coefficients, that one of them
> wasn't significant so I dropped it out of my model,
> does PROC GLM do this?

This is a problem, as it doesn't make sense to delete the coefficient of one of the 9 categorical variables that represent a single discrete variable. I would recommend that you don't do this.

Since you said that you yourself dropped the term out of the model ... yes, PROC GLM does this as well.

> Lastly,
> The Proc Glm doesn't give me estimation parameters,
> what is the synatx to get that?

In the MODEL statement, use the SOLUTION option.
N/A
Posts: 0

Re: Using Proc Reg with categorical variables

Posted in reply to deleted_user
Paige, thank you for all your help.

Last question, should I be alarmed when I get the following note:


NOTE: The X'X matrix has been found to be singular, and a generalized inverse was used to solve the
normal equations. Terms whose estimates are followed by the letter 'B' are not uniquely
estimable.
Trusted Advisor
Posts: 2,115

Re: Using Proc Reg with categorical variables

Posted in reply to deleted_user
The X'X message is generally benign if you have a CLASS statement in GLM. It is used to show all levels of the CLASS variable in the analysis. Do be concerned if there are TWO rows labeled "B" for one variable, as that represents and unplanned linear dependency.
N/A
Posts: 0

Re: Using Proc Reg with categorical variables

Posted in reply to deleted_user
I've got 9 rows with the B in it.

'Do be concerned if there are TWO rows labeled "B" for one variable, as that represents and unplanned linear dependency. '

Are you saying I should only be worried if there are two rows labeled 'B' and I don't have a class statement? Message was edited by: Jrb599
Super Contributor
Posts: 281

Re: Using Proc Reg with categorical variables

Posted in reply to deleted_user
It is not a concern if the rows are all of the dummy variables (in PROC REG) or levels (in PROC GLM) representing a discrete variable. It is, in fact, an expected outcome when dummy variables (in PROC REG) or discrete variables (in PROC GLM) are used.
N/A
Posts: 0

Re: Using Proc Reg with categorical variables

Posted in reply to deleted_user
After talking to my professor, I have decided I am going to drop just one of the dummy variables; how do I go about doing this in PROC GLM. Thanks.
Super Contributor
Posts: 281

Re: Using Proc Reg with categorical variables

Posted in reply to deleted_user
You can't drop just one dummy variable in PROC GLM.

The choice of dummy variables is done internally, so you have no control over it.

Furthermore, the results you get from the PROC GLM way of doing things produces the exact same predictions, exact same sum of squares, exact same model, etc. as any other way of creating dummy variables.

In other words, if you could drop one of the dummy variables, so instead of n dummy variables representing n levels, you now have n–1 dummy variables, the results are the same.

I am surprised your professor does not know this.
N/A
Posts: 0

Re: Using Proc Reg with categorical variables

We do get it, it's the fact that Cat9 and Cat10 have no significant difference and therefore there is no need for that term with such a high p-value. They both can be estimated by the parameter without developing a poor model.
Frequent Contributor
Posts: 77

Re: Using Proc Reg with categorical variables

Posted in reply to deleted_user
Hi Jrb599,

A point to remember. The dummy variable that is not in the model represents a reference level for the categorical variable represented by the dummy variables in the model. This was mentioned by Doc@Duke at the beginning of this thread. The reference level is the one to which all other levels are compared. Therefore, if more than one dummy variables are not in the model, then all other levels are compared to the combination of the levels these dummy variables represent.
N/A
Posts: 0

Re: Using Proc Reg with categorical variables

Posted in reply to statsplank
"Hi Jrb599,

A point to remember. The dummy variable that is not in the model represents a reference level for the categorical variable represented by the dummy variables in the model. This was mentioned by Doc@Duce at the beginning of this thread. The reference level is the one to which all other levels are compared. Therefore, if more than one dummy variables are not in the model, then all other levels are compared to the combination of the levels these dummy variables represent."

Yes I get this, this is what I'm trying to do since there is no difference between Cat9 and Cat10(my reference level).
Trusted Advisor
Posts: 2,115

Re: Using Proc Reg with categorical variables

Posted in reply to deleted_user
Jrb599,

One thing that I had forgotten, as it is so new to SAS, is the SAS 9.2 procedure GLMSELECT. It fills the gap of allowing variable selection with CLASS variables. It also produces output that allow further analyses with REG and/or GLM.

I haven't tried it, but it may help address some of the questions that you have posed here.

Doc Muhlbaier
Duke
🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 16 replies
  • 40420 views
  • 0 likes
  • 4 in conversation