turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- Analytics
- /
- Stat Procs
- /
- How to do the model validation in SAS?

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

12-02-2011 04:36 PM

Hi,

I would like to know how can I take an new dataset to validate an established model.

For example: If I have a model to investigate the relationships between health events and climatic variables in New York, how to use this model to fit the different region with the same variables?

I know it's easy to do in R but I have no idea how to do it in SAS.

Thanks a lot!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to buski

12-04-2011 09:49 PM

How about to use 'by region' statement in your proc to fit the different region?

Ksharp

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to Ksharp

12-12-2011 05:52 PM

Thanks,

But I think "by region" will give me two set of different models...not model validation

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to buski

12-04-2011 10:34 PM

With logistic regression, you can fit an existing model to new data; the syntax is in the manual. Then you can do something like a Hosmer-Lemeshow test to check the goodness of fit of the existing model to the new data.

Doc Muhlbaier

Duke

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to Doc_Duke

12-12-2011 06:02 PM

Unfortunately, I used "PROC PDL" function and the predprobs=x option does not apply.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to buski

12-12-2011 06:19 PM

But can't you still use the STB option and proc score and determine how much variance you account for with each by variable?

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to art297

12-15-2011 05:02 PM

I don't think that use STB option and Proc SCORE is equivalent to the model validation.

I am not going to evaluate the importance of different variables.

What I want to do is to use a new dataset to validate the existing model.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to buski

12-15-2011 10:47 PM

Model validation is dependent on different Estimator.

Like F estimator for proc reg and AIC for other model.

I am not sure whether I understood your Model validation.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to buski

12-15-2011 11:08 PM

Buski,

I've not used PROC PDLREG, so I'm basing these comments on it's documentation. It seems that PDLREG is linear regression with the lag variables from the time series included as an orthogonal polynomial. If my understanding is correct, then a straightforward way to get a handle on the adequacy of the model for a new population is to compute the regression estimate (y-hat) on the new data using the existing model and then use that as a single covariate in a new model that is otherwise specified exactly as the original one was. If the model is adequate, then the new coefficients will not be significant. To the extent that they are significant (beyond that expected from type I randomness), there is evidence of inadequacy.

This is not a nice clean single number as a "score", but it can be quite helpful in figuring out where the model needs work.

Doc Muhlbaier

Duke

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to buski

09-23-2012 05:27 PM

Buski, I was wondering if you figure out the way to validate a model. I am trying to do the same and having a hard time. I am trying to validate using ROC and calibration. I know how to get ROC on its own but I can't seem to figure out how to include the coefficients. Please let me know if you figured out. Thanks