topic Re: Using Proc Reg with categorical variables in SAS Procedures

Using Proc Reg with categorical variables

deleted_user — Fri, 17 Jul 2009 18:56:38 GMT

My goal is to develop a model with proc reg using both categorical (such as gender) and and continuous variables (such as age) to predict a continuous outcome (such as profit). Can proc reg do this? If not what can? If it can, do I specify which ones are categorical? Lastly, I also have a zipcode predictor which can take on 10 values, how do I change this into a binary predictor for regression?

Sorry if these questions are obvious, I'm still learning how to navigate SAS documentation.

-Thanks

Re: Using Proc Reg with categorical variables

Doc_Duke — Tue, 28 Mar 2017 11:59:49 GMT

PROC REG does not support categorical predictors directly. You have to recode them into a series of 0-1 values and use them in the model. A two-level categorical variable (like gender) becomes a simple 0-1 recode and then treated as continuous. A three-level categorical variable becomes two variables, etc.

This is analogous to the reference cell recoding that can be used in PROC GLM for categorical variables. The place that it falls down is that if you use the variable selection tools in REG, then you can end up with the situation of part of a variable in the model.

@Paige agrees:

I would use PROC GLM instead of PROC REG. Your predictor variables that are categories (gender, zip) are placed in the CLASS statement.

Also consider GLMSELECT procedure. It fills the gap of allowing variable selection with CLASS variables. It also produces output that allow further analyses with REG and/or GLM. GLMSELECT treats a class variable as a single multi-degree of freedom test for inclusion/exclusion. Either all levels are in or all levels are out; it's not a piecemeal process.

@Paige points out:

One thing that I feel needs to be pointed out here is that, despite the introduction of PROC GLMSELECT by SAS, many statisticians feel that STEPWISE (including forward and backward) model selection procedures is dangerous and misleading, and advise against using such. (Yes, I know there are other selection procedures in GLMSELECT, such as LAR and LASSO, which I have no knowledge of)

Editor's note: this response consolidates several of the helpful replies in this thread. Read through the entire topic to see the conversation.

Re: Using Proc Reg with categorical variables

Paige — Mon, 20 Jul 2009 13:52:34 GMT

I would use PROC GLM instead of PROC REG. Your predictor variables that are categories (gender, zip) are placed in the CLASS statement.

Re: Using Proc Reg with categorical variables

deleted_user — Tue, 21 Jul 2009 15:29:03 GMT

Thanks for the help.

I ran the regression with both PROC REG (created dummy variables) and PROC GLM. For the 10 values of the discrete variable, I created 9 dummy variables.

Also I noticed using proc reg that out of my 9 categorical variables coefficients, that one of them wasn't significant so I dropped it out of my model, does PROC GLM do this?

Lastly,
The Proc Glm doesn't give me estimation parameters, what is the synatx to get that?

Re: Using Proc Reg with categorical variables

Paige — Tue, 21 Jul 2009 17:05:48 GMT

> Thanks for the help.
>
> I ran the regression with both PROC REG (created
> dummy variables) and PROC GLM. For the 10 values of
> the discrete variable, I created 9 dummy variables.
>
> Also I noticed using proc reg that out of my 9
> categorical variables coefficients, that one of them
> wasn't significant so I dropped it out of my model,
> does PROC GLM do this?

This is a problem, as it doesn't make sense to delete the coefficient of one of the 9 categorical variables that represent a single discrete variable. I would recommend that you don't do this.

Since you said that you yourself dropped the term out of the model ... yes, PROC GLM does this as well.

> Lastly,
> The Proc Glm doesn't give me estimation parameters,
> what is the synatx to get that?

In the MODEL statement, use the SOLUTION option.

Re: Using Proc Reg with categorical variables

deleted_user — Tue, 21 Jul 2009 17:51:57 GMT

Paige, thank you for all your help.

Last question, should I be alarmed when I get the following note:

NOTE: The X'X matrix has been found to be singular, and a generalized inverse was used to solve the
normal equations. Terms whose estimates are followed by the letter 'B' are not uniquely
estimable.

Re: Using Proc Reg with categorical variables

Doc_Duke — Tue, 21 Jul 2009 18:33:46 GMT

The X'X message is generally benign if you have a CLASS statement in GLM. It is used to show all levels of the CLASS variable in the analysis. Do be concerned if there are TWO rows labeled "B" for one variable, as that represents and unplanned linear dependency.

Re: Using Proc Reg with categorical variables

deleted_user — Tue, 21 Jul 2009 18:47:48 GMT

I've got 9 rows with the B in it.

'Do be concerned if there are TWO rows labeled "B" for one variable, as that represents and unplanned linear dependency. '

Are you saying I should only be worried if there are two rows labeled 'B' and I don't have a class statement? Message was edited by: Jrb599

Re: Using Proc Reg with categorical variables

Paige — Tue, 21 Jul 2009 20:28:52 GMT

It is not a concern if the rows are all of the dummy variables (in PROC REG) or levels (in PROC GLM) representing a discrete variable. It is, in fact, an expected outcome when dummy variables (in PROC REG) or discrete variables (in PROC GLM) are used.

Re: Using Proc Reg with categorical variables

deleted_user — Sat, 25 Jul 2009 17:28:32 GMT

After talking to my professor, I have decided I am going to drop just one of the dummy variables; how do I go about doing this in PROC GLM. Thanks.

Re: Using Proc Reg with categorical variables

Paige — Mon, 27 Jul 2009 12:55:51 GMT

You can't drop just one dummy variable in PROC GLM.

The choice of dummy variables is done internally, so you have no control over it.

Furthermore, the results you get from the PROC GLM way of doing things produces the exact same predictions, exact same sum of squares, exact same model, etc. as any other way of creating dummy variables.

In other words, if you could drop one of the dummy variables, so instead of n dummy variables representing n levels, you now have n–1 dummy variables, the results are the same.

I am surprised your professor does not know this.

Re: Using Proc Reg with categorical variables

deleted_user — Mon, 27 Jul 2009 13:08:59 GMT

We do get it, it's the fact that Cat9 and Cat10 have no significant difference and therefore there is no need for that term with such a high p-value. They both can be estimated by the parameter without developing a poor model.

Re: Using Proc Reg with categorical variables

statsplank — Mon, 27 Jul 2009 16:37:27 GMT

Hi Jrb599,

A point to remember. The dummy variable that is not in the model represents a reference level for the categorical variable represented by the dummy variables in the model. This was mentioned by Doc@Duke at the beginning of this thread. The reference level is the one to which all other levels are compared. Therefore, if more than one dummy variables are not in the model, then all other levels are compared to the combination of the levels these dummy variables represent.

Re: Using Proc Reg with categorical variables

Doc_Duke — Mon, 27 Jul 2009 16:44:30 GMT

Jrb599,

One thing that I had forgotten, as it is so new to SAS, is the SAS 9.2 procedure GLMSELECT. It fills the gap of allowing variable selection with CLASS variables. It also produces output that allow further analyses with REG and/or GLM.

I haven't tried it, but it may help address some of the questions that you have posed here.

Doc Muhlbaier
Duke

Re: Using Proc Reg with categorical variables

deleted_user — Mon, 27 Jul 2009 17:02:25 GMT

"Hi Jrb599,

A point to remember. The dummy variable that is not in the model represents a reference level for the categorical variable represented by the dummy variables in the model. This was mentioned by Doc@Duce at the beginning of this thread. The reference level is the one to which all other levels are compared. Therefore, if more than one dummy variables are not in the model, then all other levels are compared to the combination of the levels these dummy variables represent."

Yes I get this, this is what I'm trying to do since there is no difference between Cat9 and Cat10(my reference level).

Re: Using Proc Reg with categorical variables

Paige — Mon, 27 Jul 2009 19:13:11 GMT

> One thing that I had forgotten, as it is so new to
> SAS, is the SAS 9.2 procedure GLMSELECT. It fills
> the gap of allowing variable selection with CLASS
> variables. It also produces output that allow
> further analyses with REG and/or GLM.
>
> I haven't tried it, but it may help address some of
> the questions that you have posed here.
>
> Doc Muhlbaier

One thing that I feel needs to be pointed out here is that, despite the introduction of PROC GLMSELECT by SAS, many statisticians feel that STEPWISE (including forward and backward) model selection procedures is dangerous and misleading, and advise against using such. (Yes, I know there are other selection procedures in GLMSELECT, such as LAR and LASSO, which I have no knowledge of)

Further, it's not obvious to me that model selection procedures used on dummy variables representing a single categorical variable makes any sense.

Re: Using Proc Reg with categorical variables

Doc_Duke — Mon, 27 Jul 2009 19:22:37 GMT

Paige's point about the "value" of any stepwise process is valid. It's another tool that may or may not be appropriate in a given situation.

GLMSELECT treats a class variable as a single multi-degree of freedom test for inclusion/exclusion. Either all levels are in or all levels are out; it's not a piecemeal process.