topic How to do the model validation in SAS? in Statistical Procedures

How to do the model validation in SAS?

buski — Fri, 02 Dec 2011 21:36:32 GMT

Hi,

I would like to know how can I take an new dataset to validate an established model.

For example: If I have a model to investigate the relationships between health events and climatic variables in New York, how to use this model to fit the different region with the same variables?

I know it's easy to do in R but I have no idea how to do it in SAS.

Thanks a lot!

How to do the model validation in SAS?

Ksharp — Mon, 05 Dec 2011 02:49:16 GMT

How about to use 'by region' statement in your proc to fit the different region?

Ksharp

How to do the model validation in SAS?

Doc_Duke — Mon, 05 Dec 2011 03:34:45 GMT

With logistic regression, you can fit an existing model to new data; the syntax is in the manual. Then you can do something like a Hosmer-Lemeshow test to check the goodness of fit of the existing model to the new data.

Doc Muhlbaier

Duke

How to do the model validation in SAS?

buski — Mon, 12 Dec 2011 22:52:23 GMT

Thanks,

But I think "by region" will give me two set of different models...not model validation

How to do the model validation in SAS?

buski — Mon, 12 Dec 2011 23:02:32 GMT

Unfortunately, I used "PROC PDL" function and the predprobs=x option does not apply.

How to do the model validation in SAS?

art297 — Mon, 12 Dec 2011 23:19:16 GMT

But can't you still use the STB option and proc score and determine how much variance you account for with each by variable?

How to do the model validation in SAS?

buski — Thu, 15 Dec 2011 22:02:26 GMT

I don't think that use STB option and Proc SCORE is equivalent to the model validation.

I am not going to evaluate the importance of different variables.

What I want to do is to use a new dataset to validate the existing model.

How to do the model validation in SAS?

Ksharp — Fri, 16 Dec 2011 03:47:32 GMT

Model validation is dependent on different Estimator.

Like F estimator for proc reg and AIC for other model.

I am not sure whether I understood your Model validation.

How to do the model validation in SAS?

Doc_Duke — Fri, 16 Dec 2011 04:08:48 GMT

Buski,

I've not used PROC PDLREG, so I'm basing these comments on it's documentation. It seems that PDLREG is linear regression with the lag variables from the time series included as an orthogonal polynomial. If my understanding is correct, then a straightforward way to get a handle on the adequacy of the model for a new population is to compute the regression estimate (y-hat) on the new data using the existing model and then use that as a single covariate in a new model that is otherwise specified exactly as the original one was. If the model is adequate, then the new coefficients will not be significant. To the extent that they are significant (beyond that expected from type I randomness), there is evidence of inadequacy.

This is not a nice clean single number as a "score", but it can be quite helpful in figuring out where the model needs work.

Doc Muhlbaier

Duke

Re: How to do the model validation in SAS?

Rashu — Sun, 23 Sep 2012 21:27:07 GMT

Buski, I was wondering if you figure out the way to validate a model. I am trying to do the same and having a hard time. I am trying to validate using ROC and calibration. I know how to get ROC on its own but I can't seem to figure out how to include the coefficients. Please let me know if you figured out. Thanks