topic Gini Coefficient - Variable Importance Measure in Statistical Procedures

Gini Coefficient - Variable Importance Measure

Ujjawal — Tue, 23 Jun 2015 18:37:52 GMT

There is a whitepaper for selecting important variables in a linear regression model. The URL of the whitepaper is http://support.sas.com/resources/papers/proceedings15/3242-2015.pdf .

It explains gini coefficient can be used to check linearity in the model. And we can also rank variable based on their GINI coefficient. A higher Gini coefficient suggests a higher potential for the variable to be useful in a linear regression. If a numeric variable is high on IV Rank but low on Gini coefficient , it usually suggests a lack of linearity.

My Question - Is it the gini coefficient derived from decision tree? Or it is related to Area under Curve (AUC) -- (Gini = 2*AUC- 1)? What is the exact calculation of this Gini Coefficient and how it can be used to check linearity? I googled a lot. What i got is it is used in economics theory to check inequality.

Any help would be highly appreciated. Thanks!

Re: Gini Coefficient - Variable Importance Measure

Ksharp — Wed, 24 Jun 2015 13:14:12 GMT

Check proc univariate who can calculate GINI .

Re: Gini Coefficient - Variable Importance Measure

Ujjawal — Wed, 24 Jun 2015 18:31:54 GMT

Thanks Xia. The Gini that PROC UNIVARIATE produces is a measure of statistical dispersion. Correct me if i am wrong? A low Gini coefficient indicates a more equal distribution, with 0 corresponding to complete equality. How it can be used to check linearity? How it can be used in modeling process to select important linear variables?

Re: Gini Coefficient - Variable Importance Measure

Ksharp — Thu, 25 Jun 2015 11:46:03 GMT

Sorry. I will leave it to Steve or lvm .

Re: Gini Coefficient - Variable Importance Measure

Ujjawal — Thu, 25 Jun 2015 21:01:46 GMT

Thanks Xia for looking into it.

Re: Gini Coefficient - Variable Importance Measure

SteveDenham — Fri, 26 Jun 2015 17:35:18 GMT

The Gini coefficient or Somers' D statistic gives a measure of concordance in logistic models. It is a rank based statistic, where all results are paired (all observed with all predicted). In linear regression, it is a transformation of the Pearson correlation coefficient.

Here is a nice paper that covers a lot of what is buried in the SGF paper.

http://www.imperial.ac.uk/nhli/r.newson/miscdocs/intsomd1.pdf

Steve Denham.

Re: Gini Coefficient - Variable Importance Measure

Ujjawal — Fri, 26 Jun 2015 20:55:18 GMT

Thanks a ton Steve for your answer. I know Somer's D and Gini Coefficient. Gini Coefficient = 2 (AUC -1) and AUC = %Concordance + 0.5 (Tied Pairs). It would be great if you share an article of "In linear regression, it is a transformation of the Pearson correlation coefficient.". I am more intersted about application of Gini Coefficient in linear regression. I did not find a single article to support it.

Am i correct? -

In logistic regression, if gini coefficient is high, logit function is monotonically related to independent variable?

In linear regression, if gini coefficient is high, y is linearly related to independent variable?

Re: Gini Coefficient - Variable Importance Measure

SteveDenham — Mon, 29 Jun 2015 12:51:35 GMT

Regarding correctness of interpretation, that is the way I would interpret it.

The quote is from the Imperial College paper I linked to.

Steve Denham

Re: Gini Coefficient - Variable Importance Measure

Srikanthg — Wed, 13 Jul 2016 16:00:50 GMT

What Is the best GINI cut off value for selecting the significant variable. I do understand that if gini value is high, it's a good variable in separating goods and bads.