Hi all,
Any ideas on how the gini measure of diversity is calculated when creating bins through the interactive binning node? cause i see it alters when i either merge or spliit bins
Thank you in advance
Hi, thanks to one of my R&D colleagues, he wrote up a quick solution using IML, which should match your results in EM.
proc iml;
event={517,365,349,269};
nonevent={233,281,451,535};
sum1=0;
do i=2 to 4;
sum2=0;
im1=i-1;
do j=1 to im1;
sum2=sum2+nonevent
end;
sum1=sum1 + event*sum2;
end;
sum3=0;
do i=1 to 4;
sum3 = sum3 + event*nonevent;
end;
gini = 100*(1 - (2*sum1 + sum3)/ (1500*1500)) ;
print sum1 sum2 sum3 gini;
quit;
524016 | 965 | 524340 | 30.1168 |
Hi as you may already know, the Gini statistic tests for differences in bin values with respect to the target (events vs non-events). As Gini changes in the Interactive Binning node, you get an idea of the validity of your bin cutoffs.
You can find this equation in the EM reference guide in the Interactive Binning node section.
Thanks,
Jonathan
Ok i think i am almost there, but how would the formula look like in the following example?
Value | Group | Cutoff | Event count | Non event count | Total | Event rate |
MISSING | 1 | 0 | 0 | 0 | 0 | |
AGE <25 | 2 | 25.5 | 517 | 233 | 750 | 68.9% |
25.5<AGE<31 | 3 | 31 | 365 | 281 | 646 | 56.5% |
31.5<AGE<40 | 4 | 31.5 | 349 | 451 | 800 | 43.6% |
40<AGE | 5 | 269 | 535 | 804 | 33.5% |
Thank you in advance
Hi, thanks to one of my R&D colleagues, he wrote up a quick solution using IML, which should match your results in EM.
proc iml;
event={517,365,349,269};
nonevent={233,281,451,535};
sum1=0;
do i=2 to 4;
sum2=0;
im1=i-1;
do j=1 to im1;
sum2=sum2+nonevent
end;
sum1=sum1 + event*sum2;
end;
sum3=0;
do i=1 to 4;
sum3 = sum3 + event*nonevent;
end;
gini = 100*(1 - (2*sum1 + sum3)/ (1500*1500)) ;
print sum1 sum2 sum3 gini;
quit;
524016 | 965 | 524340 | 30.1168 |
Thank you very much for your help
Don't miss out on SAS Innovate - Register now for the FREE Livestream!
Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.
Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.
Find more tutorials on the SAS Users YouTube channel.