Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

looking for macro on saving chi square differences generated by bootstrap

Accepted Solution Solved
Reply
Contributor
Posts: 49
Accepted Solution

looking for macro on saving chi square differences generated by bootstrap

[ Edited ]

I'm struggling with the methodology and technique metioned below. If there is any one who has some ideas on how this work with SAS Macro or kindly point me to the right direction to google the 'key words/terminology', that would be a great help! Thanks!

 

The purpose of doing it is to show the contribution to the predictiveness of adding the 6 selected variables.

 

1. First, generate 500 bootstrap samples of both the new model B and new model C and “freeze the samples.” 

2. For each bootstrap sample you compare the model B and model C chi-square.  There will be 500 pairs of chi square differences, which represents the population of chi-square differences.  

3. We can then compare with a chi-square test using the number of degrees of freedom of the selected variables included.  So, for example, if 6 variables are included in the final model after bootstrapping, we would run a chi square test with 6 degrees of freedom. 

 

You valuable inputs would be highly appreciated! Thank you.

 


Accepted Solutions
Solution
‎07-21-2016 11:20 AM
SAS Employee
Posts: 106

Re: looking for macro on saving chi square differences generated by bootstrap

[ Edited ]
Posted in reply to Crystal_F

Here is some code just to (hopefully) get you started. 

 

It generates a distribution of LR chisquare statistics for a logistic regression model run on 25 samples of the Cars dataset. 

 

It would need to be adapted for your situation (your data/model/modeling proc). Also, you want to compare two models and I'm running just one. 

 

proc surveyselect data=sashelp.cars out=samples
	 seed=30459584 
	 method=urs 
	 outhits
	 samprate=.50
	 rep=25; 
 run;


proc logistic data = samples ;	
    class origin; 
	model msrp = horsepower length origin; 
	by replicate;  
	ods output globaltests = LRChiSq (where=(test='Likelihood Ratio'));
run; 


proc univariate data = LRChiSq;
	var chiSq; 
	histogram;
run; 

 

View solution in original post


All Replies
SAS Employee
Posts: 106

Re: looking for macro on saving chi squre differences generated by boostrap

Posted in reply to Crystal_F

Hi. 

 

You may not need a macro for this. It can probably be done with BY group processing.

 

PROC SURVEYSELECT can create a dataset containing your 500 wr samples. 

 

Most modeling procs (proc logistic, etc.) support by groups. 

 

You can save chi-square statistics for each sample (group) using ODS Output.

 

This document may help: 

 

http://support.sas.com/resources/papers/proceedings10/268-2010.pdf

 

 

 

Ray

 

 

Solution
‎07-21-2016 11:20 AM
SAS Employee
Posts: 106

Re: looking for macro on saving chi square differences generated by bootstrap

[ Edited ]
Posted in reply to Crystal_F

Here is some code just to (hopefully) get you started. 

 

It generates a distribution of LR chisquare statistics for a logistic regression model run on 25 samples of the Cars dataset. 

 

It would need to be adapted for your situation (your data/model/modeling proc). Also, you want to compare two models and I'm running just one. 

 

proc surveyselect data=sashelp.cars out=samples
	 seed=30459584 
	 method=urs 
	 outhits
	 samprate=.50
	 rep=25; 
 run;


proc logistic data = samples ;	
    class origin; 
	model msrp = horsepower length origin; 
	by replicate;  
	ods output globaltests = LRChiSq (where=(test='Likelihood Ratio'));
run; 


proc univariate data = LRChiSq;
	var chiSq; 
	histogram;
run; 

 

Contributor
Posts: 49

Re: looking for macro on saving chi squre differences generated by boostrap

This is very helpful! Thank you so much for the detailed instruction. Really appreicat that.

SAS Employee
Posts: 106

Re: looking for macro on saving chi squre differences generated by boostrap

Posted in reply to Crystal_F

You're welcome.

 

(I just realized I used a interval dependent variable in my example. Please don't let that be a distraction. Smiley Happy)

 

Ray

Contributor
Posts: 49

Re: looking for macro on saving chi squre differences generated by boostrap

Hi Ray,

Thanks for the hint.
☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 5 replies
  • 465 views
  • 2 likes
  • 2 in conversation