looking for macro on saving chi square differences generated by bootstrap

[Crystal_F]

I'm struggling with the methodology and technique metioned below. If there is any one who has some ideas on how this work with SAS Macro or kindly point me to the right direction to google the 'key words/terminology', that would be a great help! Thanks!

The purpose of doing it is to show the contribution to the predictiveness of adding the 6 selected variables.

1. First, generate 500 bootstrap samples of both the new model B and new model C and “freeze the samples.” 

2. For each bootstrap sample you compare the model B and model C chi-square.  There will be 500 pairs of chi square differences, which represents the population of chi-square differences.  

3. We can then compare with a chi-square test using the number of degrees of freedom of the selected variables included.  So, for example, if 6 variables are included in the final model after bootstrapping, we would run a chi square test with 6 degrees of freedom. 

You valuable inputs would be highly appreciated! Thank you.

[rayIII]

Hi. 

You may not need a macro for this. It can probably be done with BY group processing.

PROC SURVEYSELECT can create a dataset containing your 500 wr samples. 

Most modeling procs (proc logistic, etc.) support by groups. 

You can save chi-square statistics for each sample (group) using ODS Output.

This document may help: 

http://support.sas.com/resources/papers/proceedings10/268-2010.pdf

Ray

[rayIII · Accepted solution]

Here is some code just to (hopefully) get you started. 

It generates a distribution of LR chisquare statistics for a logistic regression model run on 25 samples of the Cars dataset. 

It would need to be adapted for your situation (your data/model/modeling proc). Also, you want to compare two models and I'm running just one. 

proc surveyselect data=sashelp.cars out=samples
	 seed=30459584 
	 method=urs 
	 outhits
	 samprate=.50
	 rep=25; 
 run;


proc logistic data = samples ;	
    class origin; 
	model msrp = horsepower length origin; 
	by replicate;  
	ods output globaltests = LRChiSq (where=(test='Likelihood Ratio'));
run; 


proc univariate data = LRChiSq;
	var chiSq; 
	histogram;
run; 

[Crystal_F]

This is very helpful! Thank you so much for the detailed instruction. Really appreicat that.

[rayIII]

You're welcome.

(I just realized I used a interval dependent variable in my example. Please don't let that be a distraction. :))

Ray

[Crystal_F]

Hi Ray,

Thanks for the hint.