I'm struggling with the methodology and technique metioned below. If there is any one who has some ideas on how this work with SAS Macro or kindly point me to the right direction to google the 'key words/terminology', that would be a great help! Thanks!
The purpose of doing it is to show the contribution to the predictiveness of adding the 6 selected variables.
1. First, generate 500 bootstrap samples of both the new model B and new model C and “freeze the samples.”
2. For each bootstrap sample you compare the model B and model C chi-square. There will be 500 pairs of chi square differences, which represents the population of chi-square differences.
3. We can then compare with a chi-square test using the number of degrees of freedom of the selected variables included. So, for example, if 6 variables are included in the final model after bootstrapping, we would run a chi square test with 6 degrees of freedom.
You valuable inputs would be highly appreciated! Thank you.
Here is some code just to (hopefully) get you started.
It generates a distribution of LR chisquare statistics for a logistic regression model run on 25 samples of the Cars dataset.
It would need to be adapted for your situation (your data/model/modeling proc). Also, you want to compare two models and I'm running just one.
proc surveyselect data=sashelp.cars out=samples
seed=30459584
method=urs
outhits
samprate=.50
rep=25;
run;
proc logistic data = samples ;
class origin;
model msrp = horsepower length origin;
by replicate;
ods output globaltests = LRChiSq (where=(test='Likelihood Ratio'));
run;
proc univariate data = LRChiSq;
var chiSq;
histogram;
run;
Hi.
You may not need a macro for this. It can probably be done with BY group processing.
PROC SURVEYSELECT can create a dataset containing your 500 wr samples.
Most modeling procs (proc logistic, etc.) support by groups.
You can save chi-square statistics for each sample (group) using ODS Output.
This document may help:
http://support.sas.com/resources/papers/proceedings10/268-2010.pdf
Ray
Here is some code just to (hopefully) get you started.
It generates a distribution of LR chisquare statistics for a logistic regression model run on 25 samples of the Cars dataset.
It would need to be adapted for your situation (your data/model/modeling proc). Also, you want to compare two models and I'm running just one.
proc surveyselect data=sashelp.cars out=samples
seed=30459584
method=urs
outhits
samprate=.50
rep=25;
run;
proc logistic data = samples ;
class origin;
model msrp = horsepower length origin;
by replicate;
ods output globaltests = LRChiSq (where=(test='Likelihood Ratio'));
run;
proc univariate data = LRChiSq;
var chiSq;
histogram;
run;
This is very helpful! Thank you so much for the detailed instruction. Really appreicat that.
You're welcome.
(I just realized I used a interval dependent variable in my example. Please don't let that be a distraction. :))
Ray
Build your skills. Make connections. Enjoy creative freedom. Maybe change the world. Registration is now open through August 30th. Visit the SAS Hackathon homepage.
Register today!Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.
Find more tutorials on the SAS Users YouTube channel.