BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
My goal is to develop a model with proc reg using both categorical (such as gender) and and continuous variables (such as age) to predict a continuous outcome (such as profit). Can proc reg do this? If not what can? If it can, do I specify which ones are categorical? Lastly, I also have a zipcode predictor which can take on 10 values, how do I change this into a binary predictor for regression?

Sorry if these questions are obvious, I'm still learning how to navigate SAS documentation.

-Thanks
1 ACCEPTED SOLUTION

Accepted Solutions
Doc_Duke
Rhodochrosite | Level 12

PROC REG does not support categorical predictors directly. You have to recode them into a series of 0-1 values and use them in the model. A two-level categorical variable (like gender) becomes a simple 0-1 recode and then treated as continuous. A three-level categorical variable becomes two variables, etc.

This is analogous to the reference cell recoding that can be used in PROC GLM for categorical variables. The place that it falls down is that if you use the variable selection tools in REG, then you can end up with the situation of part of a variable in the model.

 

@Paige agrees:

I would use PROC GLM instead of PROC REG. Your predictor variables that are categories (gender, zip) are placed in the CLASS statement.

Also consider GLMSELECT procedure.   It fills the gap of allowing variable selection with CLASS variables. It also produces output that allow further analyses with REG and/or GLM.  GLMSELECT treats a class variable as a single multi-degree of freedom test for inclusion/exclusion. Either all levels are in or all levels are out; it's not a piecemeal process.

 

@Paige points out:

One thing that I feel needs to be pointed out here is that, despite the introduction of PROC GLMSELECT by SAS, many statisticians feel that STEPWISE (including forward and backward) model selection procedures is dangerous and misleading, and advise against using such. (Yes, I know there are other selection procedures in GLMSELECT, such as LAR and LASSO, which I have no knowledge of)

Editor's note: this response consolidates several of the helpful replies in this thread.  Read through the entire topic to see the conversation.

View solution in original post

16 REPLIES 16
Doc_Duke
Rhodochrosite | Level 12

PROC REG does not support categorical predictors directly. You have to recode them into a series of 0-1 values and use them in the model. A two-level categorical variable (like gender) becomes a simple 0-1 recode and then treated as continuous. A three-level categorical variable becomes two variables, etc.

This is analogous to the reference cell recoding that can be used in PROC GLM for categorical variables. The place that it falls down is that if you use the variable selection tools in REG, then you can end up with the situation of part of a variable in the model.

 

@Paige agrees:

I would use PROC GLM instead of PROC REG. Your predictor variables that are categories (gender, zip) are placed in the CLASS statement.

Also consider GLMSELECT procedure.   It fills the gap of allowing variable selection with CLASS variables. It also produces output that allow further analyses with REG and/or GLM.  GLMSELECT treats a class variable as a single multi-degree of freedom test for inclusion/exclusion. Either all levels are in or all levels are out; it's not a piecemeal process.

 

@Paige points out:

One thing that I feel needs to be pointed out here is that, despite the introduction of PROC GLMSELECT by SAS, many statisticians feel that STEPWISE (including forward and backward) model selection procedures is dangerous and misleading, and advise against using such. (Yes, I know there are other selection procedures in GLMSELECT, such as LAR and LASSO, which I have no knowledge of)

Editor's note: this response consolidates several of the helpful replies in this thread.  Read through the entire topic to see the conversation.

Paige
Quartz | Level 8
I would use PROC GLM instead of PROC REG. Your predictor variables that are categories (gender, zip) are placed in the CLASS statement.
deleted_user
Not applicable
Thanks for the help.

I ran the regression with both PROC REG (created dummy variables) and PROC GLM. For the 10 values of the discrete variable, I created 9 dummy variables.

Also I noticed using proc reg that out of my 9 categorical variables coefficients, that one of them wasn't significant so I dropped it out of my model, does PROC GLM do this?

Lastly,
The Proc Glm doesn't give me estimation parameters, what is the synatx to get that?
Paige
Quartz | Level 8
> Thanks for the help.
>
> I ran the regression with both PROC REG (created
> dummy variables) and PROC GLM. For the 10 values of
> the discrete variable, I created 9 dummy variables.
>
> Also I noticed using proc reg that out of my 9
> categorical variables coefficients, that one of them
> wasn't significant so I dropped it out of my model,
> does PROC GLM do this?

This is a problem, as it doesn't make sense to delete the coefficient of one of the 9 categorical variables that represent a single discrete variable. I would recommend that you don't do this.

Since you said that you yourself dropped the term out of the model ... yes, PROC GLM does this as well.

> Lastly,
> The Proc Glm doesn't give me estimation parameters,
> what is the synatx to get that?

In the MODEL statement, use the SOLUTION option.
deleted_user
Not applicable
Paige, thank you for all your help.

Last question, should I be alarmed when I get the following note:


NOTE: The X'X matrix has been found to be singular, and a generalized inverse was used to solve the
normal equations. Terms whose estimates are followed by the letter 'B' are not uniquely
estimable.
Doc_Duke
Rhodochrosite | Level 12
The X'X message is generally benign if you have a CLASS statement in GLM. It is used to show all levels of the CLASS variable in the analysis. Do be concerned if there are TWO rows labeled "B" for one variable, as that represents and unplanned linear dependency.
deleted_user
Not applicable
I've got 9 rows with the B in it.

'Do be concerned if there are TWO rows labeled "B" for one variable, as that represents and unplanned linear dependency. '

Are you saying I should only be worried if there are two rows labeled 'B' and I don't have a class statement? Message was edited by: Jrb599
Paige
Quartz | Level 8
It is not a concern if the rows are all of the dummy variables (in PROC REG) or levels (in PROC GLM) representing a discrete variable. It is, in fact, an expected outcome when dummy variables (in PROC REG) or discrete variables (in PROC GLM) are used.
deleted_user
Not applicable
After talking to my professor, I have decided I am going to drop just one of the dummy variables; how do I go about doing this in PROC GLM. Thanks.
Paige
Quartz | Level 8
You can't drop just one dummy variable in PROC GLM.

The choice of dummy variables is done internally, so you have no control over it.

Furthermore, the results you get from the PROC GLM way of doing things produces the exact same predictions, exact same sum of squares, exact same model, etc. as any other way of creating dummy variables.

In other words, if you could drop one of the dummy variables, so instead of n dummy variables representing n levels, you now have n–1 dummy variables, the results are the same.

I am surprised your professor does not know this.
deleted_user
Not applicable
We do get it, it's the fact that Cat9 and Cat10 have no significant difference and therefore there is no need for that term with such a high p-value. They both can be estimated by the parameter without developing a poor model.
statsplank
Calcite | Level 5
Hi Jrb599,

A point to remember. The dummy variable that is not in the model represents a reference level for the categorical variable represented by the dummy variables in the model. This was mentioned by Doc@Duke at the beginning of this thread. The reference level is the one to which all other levels are compared. Therefore, if more than one dummy variables are not in the model, then all other levels are compared to the combination of the levels these dummy variables represent.
deleted_user
Not applicable
"Hi Jrb599,

A point to remember. The dummy variable that is not in the model represents a reference level for the categorical variable represented by the dummy variables in the model. This was mentioned by Doc@Duce at the beginning of this thread. The reference level is the one to which all other levels are compared. Therefore, if more than one dummy variables are not in the model, then all other levels are compared to the combination of the levels these dummy variables represent."

Yes I get this, this is what I'm trying to do since there is no difference between Cat9 and Cat10(my reference level).
Doc_Duke
Rhodochrosite | Level 12
Jrb599,

One thing that I had forgotten, as it is so new to SAS, is the SAS 9.2 procedure GLMSELECT. It fills the gap of allowing variable selection with CLASS variables. It also produces output that allow further analyses with REG and/or GLM.

I haven't tried it, but it may help address some of the questions that you have posed here.

Doc Muhlbaier
Duke

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 16 replies
  • 103366 views
  • 8 likes
  • 4 in conversation