topic Re: Difference between proc genmod and proc hpgenselect in Statistical Procedures

Difference between proc genmod and proc hpgenselect

farazs1 — Wed, 05 Aug 2020 17:49:35 GMT

Why would I be able to run same model (log binomial with same input and output variables) in proc hpgenselect but not in proc genmod? There is no difference in my code. Class and model statements are exactly the same. Distribution and link functions are same of course. Any idea why I can run proc hpgenselect but not proc genmod?

Re: Difference between proc genmod and proc hpgenselect

PaigeMiller — Wed, 05 Aug 2020 17:59:07 GMT

What makes you believe you can't run it in PROC GENMOD? Is there an error message? If so, please show us the log for the entire PROC GENMOD step, including code, NOTEs, WARNINGs and ERRORs. Or, is there some other problem? Show us.

Re: Difference between proc genmod and proc hpgenselect

farazs1 — Wed, 05 Aug 2020 18:06:21 GMT

proc genmod data = data descending;
 class a(ref='0') b(ref='Non-Blast') c(ref='0') d(ref='0')/param=ref;
 model outcome(ref='No') = a b c d / dist = binomial link = log;
run;

Error I get from this is:

NOTE: PROC GENMOD is modeling the probability that outcome='YES'.
WARNING: The specified model did not converge.
NOTE: The Pearson chi-square and deviance are not computed since the AGGREGATE option is not
specified.
ERROR: The mean parameter is either invalid or at a limit of its range for some observations.
NOTE: The scale parameter was held fixed.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE GENMOD used (Total process time):
real time 0.09 seconds
cpu time 0.06 seconds

As compared to proc hpgenselect:

PROC HPGENSELECT data=data;

CLASS a(ref='0') b(ref='Non-Blast') c(ref='0') d(ref='0')/param=ref;

model outcome(ref='No')=a b c d /dist=binomial link=log;

RUN;

log shows -

NOTE: The HPGENSELECT procedure is executing in single-machine mode.
NOTE: You are modeling the probability that outcome='YES'.
NOTE: Convergence criterion (FCONV=2.1073424E-8) satisfied.
NOTE: There were 341 observations read from the data set WORK.DATA.
NOTE: PROCEDURE HPGENSELECT used (Total process time):
real time 0.09 seconds
cpu time 0.04 seconds

Am I doing something incorrect?

Re: Difference between proc genmod and proc hpgenselect

SAS_Rob — Wed, 05 Aug 2020 18:31:57 GMT

There are differences in the fitting algorithms between the two procedures such that in certain situations you will get slightly different results or convergence versus non-convergence.

If you were to adjust the convergence criterion and fitting algorithm in HPGENSELECT so that it was identical to GENMOD, I suspect you would get the same message.

PROC HPGENSELECT data=data gconv=1e-4 tech=newrap;

Unfortunately there is not a way to exactly implement what HPGENSELECT does by default in GENMOD.

Re: Difference between proc genmod and proc hpgenselect

farazs1 — Wed, 05 Aug 2020 18:50:48 GMT

Can I modify proc genmod in a way so that I can run it same as proc hpgenselect? I want to generate RR estimates off of this model but I can only get coefficients from proc hpgenselect which I will have to turn into RR estimates I want. if I can run this same model in proc genmod, then I can use estimate statement and get RR estimates. or if you have a different solution, that is also fine.

Re: Difference between proc genmod and proc hpgenselect

Ksharp — Thu, 06 Aug 2020 12:14:20 GMT

Try :
proc genmod data = data ;
class a(ref='0') b(ref='Non-Blast') c(ref='0') d(ref='0')/param=ref;
model outcome = a b c d / dist = binomial link = log;
run;

Re: Difference between proc genmod and proc hpgenselect

StatDave — Thu, 06 Aug 2020 13:06:46 GMT

Since the log link does not ensure that predicted values are valid probabilities, this error is common when fitting the log-linked binomial model. Fortunately, it isn't necessary to fit a log-linked binomial model in order to estimate relative risks. You can fit an ordinary logistic model using PROC LOGISTIC and then use the NLMeans macro to produce the relative risk estimates. See this note.

Re: Difference between proc genmod and proc hpgenselect

farazs1 — Thu, 06 Aug 2020 17:35:38 GMT

I am attempting this approach.

Unfortunately NLmeans macro is not working at the moment. I will keep at it and see what I get.

Thank you!

Re: Difference between proc genmod and proc hpgenselect

farazs1 — Thu, 06 Aug 2020 17:36:22 GMT

I believe I already tried that and it did not work.

Re: Difference between proc genmod and proc hpgenselect

StatDave — Thu, 06 Aug 2020 19:02:06 GMT

I'm not aware of anything that would keep the LOGISTIC/NLMeans approach from working. Try it and post your code and any errors you get.

Re: Difference between proc genmod and proc hpgenselect

farazs1 — Fri, 07 Aug 2020 19:13:48 GMT

I tried one of the terms, let's say term a by itself using proc logistic and then running nlmixed macro. I also ran log-binomial model using proc genmod. Relative risk that I am getting from proc genmod is vastly different from the one I get using nlmixed approach. That should not be the case, right? I can understand if it off by decimals but proc genmod is giving me almost double the RR of the other approach. Am I missing something?

Re: Difference between proc genmod and proc hpgenselect

StatDave — Fri, 07 Aug 2020 19:23:46 GMT

If you will post the code you used, I might be able to tell you what is wrong. You should just have a PROC LOGISTIC step followed by a call of the NLMeans macro. Show any log messages that they generated.

Re: Difference between proc genmod and proc hpgenselect

farazs1 — Mon, 10 Aug 2020 02:30:46 GMT

OK so I was running something wrong actually. I was modeling wrong event in one method. I am finally getting same RR for the two methods. However, 95% CL is different for each. NLmeans macro is giving me tighter bounds. Any opinion on why would this be?

Re: Difference between proc genmod and proc hpgenselect

farazs1 — Mon, 10 Aug 2020 13:22:22 GMT

any idea how to run two or more terms in one model using this macro? thank you!

Re: Difference between proc genmod and proc hpgenselect

StatDave — Mon, 10 Aug 2020 14:39:41 GMT

GENMOD and NLMeans use different valid methods to compute confidence limits. In GENMOD, the ESTIMATE statement forms a confidence interval on the specified estimate and then exponentiates the estimate and its limits. This results in an interval that is not symmetric around the relative risk estimate. NLMeans forms a confidence interval directly around the relative risk estimate and is symmetric around the estimate. It uses the delta method to estimate the standard error of the relative risk. Since the two methods are different, the widths of the intervals will differ somewhat.

The NLMeans macro provides differences or ratios for one variable. If you want to estimate the relative risk for more than one variable, you just need to run the macro more than once.

Re: Difference between proc genmod and proc hpgenselect

StatDave — Mon, 10 Aug 2020 15:06:57 GMT

For instance, if you use the log binomial method for the first example (gender and question) in note 23003, the confidence interval on the relative risk estimate is (1.0365, 2.1707) which is not centered around the relative risk estimate (1.5) and has width 1.1342 as compared to the NLMeans/delta method width 1.1087.

And sorry but I forgot that the NLMeans macro can handle multiple sets of estimates, so it can provide difference or ratio estimates for more than one variable at a time. You can specify more than one variable in the LSMEANS statement (or more than one LSMEANS statement) in the preceding modeling procedure and the macro will, by default (options=nojoint), compute differences or ratios separately for each set of LSMEANS estimates.

Re: Difference between proc genmod and proc hpgenselect

farazs1 — Mon, 10 Aug 2020 15:14:32 GMT

thanks for the quick response.

I think I will just run my model (that has more than one term) with proc logistic and convert odds ratio to relative risk. For models with one term, I will use the NLmeans approach. It would have been nice to use the same approach for models with multiple terms.

Re: Difference between proc genmod and proc hpgenselect

farazs1 — Mon, 10 Aug 2020 15:17:01 GMT

oops just saw your second post. let me try that. thank you!

Re: Difference between proc genmod and proc hpgenselect

farazs1 — Mon, 10 Aug 2020 16:55:20 GMT

I tried a new model with two terms in it.

Please see my code, log, and output for each approach.

Code –
proc genmod data = data descending;
 class var_a(ref='0') var_b(ref='0')/param=ref;
 model outcome(ref='0') = var_a var_b/ dist = binomial link = log type3;
 estimate 'Beta' var_a 1 -1/ exp;
 estimate 'Beta' var_b 1 -1/exp;
run;
Log –

NOTE: PROC GENMOD is modeling the probability that outcome='1'.
NOTE: The Pearson chi-square and deviance are not computed since the AGGREGATE option is not
      specified.
NOTE: Algorithm converged.
NOTE: The scale parameter was held fixed.
WARNING: More coefficients than levels specified for effect var_a.  Some coefficients will be
         ignored.
WARNING: More coefficients than levels specified for effect var_b.  Some coefficients will be
         ignored.
NOTE: PROCEDURE GENMOD used (Total process time):
      real time           1.15 seconds
      cpu time            0.78 seconds

output -

using nlmeans approach

proc logistic data=data;
        class var_a(ref="0") var_b(ref="0")/ param=glm;
        model outcome(event="1")=var_a var_b;
        lsmeans var_a/ e ilink;
		lsmeans var_b/ e ilink;
        ods output coef=coeffs;
        store out=ques;
        run;
		%NLMeans(instore=ques, coef=coeffs, link=logit, options=ratio, title=Relative Risk);
Log – 
NOTE: PROC LOGISTIC is modeling the probability that outcome='1'.
NOTE: Convergence criterion (GCONV=1E-8) satisfied.
NOTE: The LOGISTIC procedure generated the model item store WORK.QUES.
NOTE: The data set WORK.COEFFS has 10 observations and 6 variables.
NOTE: There were 341 observations read from the data set WORK.DATA.
NOTE: PROCEDURE LOGISTIC used (Total process time):
      real time           2.42 seconds
      cpu time            1.62 seconds


2633
2634          %NLMeans(instore=ques, coef=coeffs, link=logit, options=ratio, title=Relative Risk);
WARNING: The final Hessian matrix is not positive definite, and therefore the estimated covariance
         matrix is not full rank and may be unreliable.  The variance of some parameter estimates
         is zero or some parameters are linearly related to other parameters.
WARNING: The final Hessian matrix is not positive definite, and therefore the estimated covariance
         matrix is not full rank and may be unreliable.  The variance of some parameter estimates
         is zero or some parameters are linearly related to other parameters.
NOTE: The NLMEANS macro used 3.73 seconds.

output -

I am supposed to write separate LSMEANS statement and one macro statement, right? What am I doing wrong here?

Re: Difference between proc genmod and proc hpgenselect

StatDave — Mon, 10 Aug 2020 19:20:39 GMT

You should specify only as many coefficients for a model effect in the ESTIMATE statement as it has parameter estimates. With reference parameterization (PARAM=REF), a binary variable has only one parameter estimate, so you can specify only one coefficient. If you just change PARAM=REF to PARAM=GLM, then both binary predictors will have two parameter estimates (the last being zero in each case) and your ESTIMATE statements will not produce Warnings - though I think the results will be the same.