topic Re: Analyzing positively skewed continuous outcome variable in Statistical Procedures

Analyzing positively skewed continuous outcome variable

chepchep — Fri, 05 Mar 2021 13:21:24 GMT

Hello,

I am trying to model a continuous outcome variable which is highly skewed. I have several predictor variables in the model both continuous and categorical. The q-q plot of the residuals is shown below. As you can see, the normality assumption is clearly violated. I tried log transforming the outcome variable but it doesn't seem to fix the problem. Any body has an idea of how to remedy this issue ? Does the central limit theorem apply here?

Thanks.

Here is the code used:

proc glmselect data=b;
class a b c d e / param=reference;
model y=a b c d e f ;
output out=check r=residuals;
run;

proc univariate data=check;
var residuals;
histogram residuals / normal kernel;
qqplot residuals / normal(mu=est sigma=est);
run;

Re: Analyzing positively skewed continuous outcome variable

StatDave — Fri, 05 Mar 2021 14:19:39 GMT

Positively valued and skewed responses are often modeled using the gamma or inverse gaussian distribution as are available with the DIST= option in PROC GENMOD.

Re: Analyzing positively skewed continuous outcome variable

PGStats — Fri, 05 Mar 2021 18:01:52 GMT

What does the histogram of the residuals look like? Is there more than one mode? This would signal that you are missing some important effect, or some important interaction(s).

Re: Analyzing positively skewed continuous outcome variable

chepchep — Fri, 05 Mar 2021 18:20:45 GMT

This is how the histogram looks like:

Re: Analyzing positively skewed continuous outcome variable

chepchep — Fri, 05 Mar 2021 18:22:40 GMT

Thanks for your input. So how do I know which of the two to use, can either one of them work?

Re: Analyzing positively skewed continuous outcome variable

Rick_SAS — Fri, 05 Mar 2021 18:31:14 GMT

Personally, I would work on developing a better model. Use regression diagnostic plots to analyze whether you should include second-order interaction terms in the model. Since you are using PROC GLMSELECT, you can add in all second-order terms and use variable selection to see if any interactions improve the fit enough to make it into the final model.

Re: Analyzing positively skewed continuous outcome variable

StatDave — Fri, 05 Mar 2021 18:59:00 GMT

You can use PROC SEVERITY in SAS/ETS to assess the fit of several distributions, including gamma and inverse gaussian and others. For example:

proc severity data=b crit=aicc;
   loss y;
   dist _predefined_;
run;

Re: Analyzing positively skewed continuous outcome variable

chepchep — Mon, 08 Mar 2021 15:47:15 GMT

Thank you so much for your input. So I did use the proc severity to select the the distribution that best fits my data and the Burr distribution was selected. This is a distribution that I am not very familiar with. How do I fit a Burr distribution in SAS?

Here is the partial output from proc severity:

Distribution Converged AICC SelectedBurrExpGammaIgaussLognParetoGpdWeibull

Yes	361869	Yes
Yes	382973	No
Yes	371532	No
Yes	368401	No
Yes	365528	No
Yes	382977	No
Yes	382975	No
Yes	377067	No

Re: Analyzing positively skewed continuous outcome variable

StatDave — Mon, 08 Mar 2021 16:00:55 GMT

I suggest you look at the plots (CDF/EDF and PDF) to visually assess how close the other distributions are to the EDF of the observed data. It's not so much a matter of picking the one with the lowest AICC as it is rejecting distributions that clearly don't fit well and picking one that does fit reasonably well.

Re: Analyzing positively skewed continuous outcome variable

chepchep — Mon, 08 Mar 2021 16:34:48 GMT

Thank you so much!