12-02-2011 04:36 PM
I would like to know how can I take an new dataset to validate an established model.
For example: If I have a model to investigate the relationships between health events and climatic variables in New York, how to use this model to fit the different region with the same variables?
I know it's easy to do in R but I have no idea how to do it in SAS.
Thanks a lot!
12-04-2011 10:34 PM
With logistic regression, you can fit an existing model to new data; the syntax is in the manual. Then you can do something like a Hosmer-Lemeshow test to check the goodness of fit of the existing model to the new data.
12-15-2011 05:02 PM
I don't think that use STB option and Proc SCORE is equivalent to the model validation.
I am not going to evaluate the importance of different variables.
What I want to do is to use a new dataset to validate the existing model.
12-15-2011 10:47 PM
Model validation is dependent on different Estimator.
Like F estimator for proc reg and AIC for other model.
I am not sure whether I understood your Model validation.
12-15-2011 11:08 PM
I've not used PROC PDLREG, so I'm basing these comments on it's documentation. It seems that PDLREG is linear regression with the lag variables from the time series included as an orthogonal polynomial. If my understanding is correct, then a straightforward way to get a handle on the adequacy of the model for a new population is to compute the regression estimate (y-hat) on the new data using the existing model and then use that as a single covariate in a new model that is otherwise specified exactly as the original one was. If the model is adequate, then the new coefficients will not be significant. To the extent that they are significant (beyond that expected from type I randomness), there is evidence of inadequacy.
This is not a nice clean single number as a "score", but it can be quite helpful in figuring out where the model needs work.
09-23-2012 05:27 PM
Buski, I was wondering if you figure out the way to validate a model. I am trying to do the same and having a hard time. I am trying to validate using ROC and calibration. I know how to get ROC on its own but I can't seem to figure out how to include the coefficients. Please let me know if you figured out. Thanks