Programming the statistical procedures from SAS

How to do the model validation in SAS?

Reply
Occasional Contributor
Posts: 12

How to do the model validation in SAS?

Hi,

I would like to know how can I take an new dataset to validate an established model.

For example: If I have a model to investigate the relationships between health events and climatic variables in New York, how to use this model to fit the different region with the same variables?

I know it's easy to do in R but I have no idea how to do it in SAS.

Thanks a lot!

Super User
Posts: 9,769

How to do the model validation in SAS?

How about to use 'by region' statement in your proc to fit the different region?

Ksharp

Occasional Contributor
Posts: 12

How to do the model validation in SAS?

Thanks,

But I think "by region" will give me two set of different models...not model validation 

Trusted Advisor
Posts: 2,114

How to do the model validation in SAS?

With logistic regression, you can fit an existing model to new data; the syntax is in the manual.  Then you can do something like a Hosmer-Lemeshow test to check the goodness of fit of the existing model to the new data.

Doc Muhlbaier

Duke

Occasional Contributor
Posts: 12

How to do the model validation in SAS?

Unfortunately, I used  "PROC PDL" function and the predprobs=x option does not apply.

PROC Star
Posts: 7,416

How to do the model validation in SAS?

But can't you still use the STB option and proc score and determine how much variance you account for with each by variable?

Occasional Contributor
Posts: 12

How to do the model validation in SAS?

I don't think that use STB option and Proc SCORE is equivalent to the model validation.

I am not going to evaluate the importance of different variables.

What I want to do is to use a new dataset to validate the existing model.

Super User
Posts: 9,769

How to do the model validation in SAS?

Model validation is dependent on different Estimator.

Like F estimator for proc reg and AIC for other model.

I am not sure whether I understood your Model validation.

Trusted Advisor
Posts: 2,114

How to do the model validation in SAS?

Buski,

I've not used PROC PDLREG, so I'm basing these comments on it's documentation.  It seems that PDLREG is linear regression with the lag variables from the time series included as an orthogonal polynomial.  If my understanding is correct, then a straightforward way to get a handle on the adequacy of the model for a new population is to compute the regression estimate (y-hat) on the new data using the existing model and then use that as a single covariate in a new model that is otherwise specified exactly as the original one was.  If the model is adequate, then the new coefficients will not be significant.  To the extent that they are significant (beyond that expected from type I randomness), there is evidence of inadequacy. 

This is not a nice clean single number as a "score", but it can be quite helpful in figuring out where the model needs work.

Doc Muhlbaier

Duke

Occasional Contributor
Posts: 7

Re: How to do the model validation in SAS?

Buski, I was wondering if you figure out the way to validate a model. I am trying to do the same and having a hard time. I am trying to validate using ROC and calibration. I know how to get ROC on its own but I can't seem to figure out how to include the coefficients. Please let me know if you figured out. Thanks

Ask a Question
Discussion stats
  • 9 replies
  • 2500 views
  • 0 likes
  • 5 in conversation