topic Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur in New SAS User

PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

halkyos — Fri, 27 Sep 2019 19:45:52 GMT

I am working with risk and protective factor data for outcomes regarding substance use. My data is arranged so that I have an outcome as a binary variable (0=no use, 1= use), the total number of risk factors and the total number of protective factors. Risk factors are known to increase the likelihood of an outcome occurring and protective factors are known to have an opposite effect. Examination in PROC FREQ shows that the proportion of observations using a substance increases with the number of risk factors and decreases with the number of protective factors. When I use PROC LOGISTIC though to write a model, I am getting a positive effect from my protective factors. Here is my code:

PROC LOGISTIC DATA=survey DESCENDING;
  MODEL sub1= rfs pfs;
RUN;

sub1: binary variable where 1= using the substance and 0=not using the substance.

rfs: total number of risk factors.

pfs: total number of protective factors.

My results for one substance are giving me a model of p(1)=-3.1860+0.3033(rfs)+0.1181(pfs). As a researcher I know that this is wrong, I don't have anomalous data where the population is more likely to use substances if they have more protective factors, but I am having trouble figuring out how to correct this.

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

Reeza — Fri, 27 Sep 2019 19:48:33 GMT

Can you post your log?
And are rfs and pfs continuous or categorical?

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

PaigeMiller — Fri, 27 Sep 2019 19:49:44 GMT

One reason this can occur is if your two x-variables rfs and pfs are highly correlated with each other. Another reason this can occur is if you have outliers or clusters in rfs and/or pfs.

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

halkyos — Fri, 27 Sep 2019 20:00:13 GMT

Here is my log:

306 PROC LOGISTIC DATA=survey DESCENDING;
307 MODEL sub1=rfs pfs;
308 RUN;

NOTE: PROC LOGISTIC is modeling the probability that sub1=1.
NOTE: Convergence criterion (GCONV=1E-8) satisfied.
NOTE: There were 14445 observations read from the data set WORK.SURVEY.
NOTE: PROCEDURE LOGISTIC used (Total process time):
real time 0.25 seconds
cpu time 0.15 seconds

rfs and pfs are positive whole integers unless they are 0 representing the number of risk or protective factors present. rfs ranges from 0-21 and pfs ranges from 0-12.

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

halkyos — Fri, 27 Sep 2019 20:41:33 GMT

I just tested for these possibilities: A chi-square test for independence indicates that rfs and pfs are independent. When entered into PROC AUTOREG for rfs=pfs and pfs=rfs the values are negatively correlated to each other:

chi-sq: 4876.6114, p<0.0001.

rfs=12.9714-0.7614(pfs), p<0.0001.

pfs=8.7403-0.2989(rfs), p<0.0001.

The overall distributions are almost textbook normal, and when stratified to whether or not the observation reported substance use the distribution of rfs for non substance users takes on a right-tail skew. All other distributions remain normal.

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

Reeza — Fri, 27 Sep 2019 20:59:02 GMT

P<0.0001 means related not indepedent, doesn't it?

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

PaigeMiller — Fri, 27 Sep 2019 21:14:55 GMT

@halkyos wrote:

I just tested for these possibilities: A chi-square test for independence indicates that rfs and pfs are independent. When entered into PROC AUTOREG for rfs=pfs and pfs=rfs the values are negatively correlated to each other:

chi-sq: 4876.6114, p<0.0001.

rfs=12.9714-0.7614(pfs), p<0.0001.

pfs=8.7403-0.2989(rfs), p<0.0001.

The overall distributions are almost textbook normal, and when stratified to whether or not the observation reported substance use the distribution of rfs for non substance users takes on a right-tail skew. All other distributions remain normal.

What is the correlation (not the auto-correlation from PROC AUTOREG but the correlation from PROC CORR) between rfs and pfs? Distribution of your x-variables is irrelevant here. Are there outliers or clusters among your x-variables?

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

Ksharp — Sat, 28 Sep 2019 11:57:50 GMT

Change your response value which model the prob ,and you get the different result

PROC LOGISTIC DATA=survey ;  MODEL sub1(event='0')  = rfs pfs;RUN;

V.S.

PROC LOGISTIC DATA=survey ;  MODEL sub1(event='1')= rfs pfs;RUN;

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

Ksharp — Sat, 28 Sep 2019 12:06:56 GMT

Another possible reason is BAD data.

Check the standard error of these two positive estimate coefficient , and see if it was very big .

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

halkyos — Mon, 30 Sep 2019 14:01:36 GMT

So I tried this before coming onto here, what is does is switches which of the two has a larger positive coefficient, but both remain positive. My office is renewing my license today so I can't currently give you the exact coefficients, but what happens is it becomes sub1=y+pfs+rfs where the coefficient of pfs> coefficient of rfs; 0<= either coefficient <= 1.

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

Ksharp — Tue, 01 Oct 2019 04:34:48 GMT

Did you Check the standard error of these two coefficient ?

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

PaigeMiller — Tue, 01 Oct 2019 11:04:39 GMT

What is the correlation (not the auto-correlation from PROC AUTOREG but the correlation from PROC CORR) between rfs and pfs?

Are there outliers or clusters among your x-variables?

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

Reeza — Tue, 01 Oct 2019 14:50:40 GMT

Can you show a PROC FREQ of rfs*pfs I suspect you have some massive imbalances.

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

halkyos — Tue, 01 Oct 2019 15:41:56 GMT

The standard errors are as follows:

rfs: 0.0417

pfs: 0.0261

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

halkyos — Tue, 01 Oct 2019 15:47:12 GMT

PROC CORR is new to me, but looking at the guide on that one it seems pretty straightforward. I ran:

PROC CORR DATA=survey;
	VAR rfs pfs;
RUN;

My results are:

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

halkyos — Tue, 01 Oct 2019 15:48:32 GMT

The standard errors on the coefficients in the model are 0.00563 (rfs) and 0.00743 (pfs).

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

halkyos — Tue, 01 Oct 2019 15:54:39 GMT

My PROC CORR results are as follows:

There are no high or low outliers for either variable.

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

halkyos — Tue, 01 Oct 2019 16:10:31 GMT

Some additional exploration of the that may help us figure this out:

I decided to run four separate models:

Substance use predicted by risk factors only
Substance use predicted by protective factors only
No substance use predicted by risk factors only
No substance use predicted by protective factors only

The results are:

Substance use 'yes'=-2.1091+0.2623(rfs)
Substance use 'yes'=0.5846-0.0937(pfs)
Substance use 'no'=2.1091-0.2623(rfs)
Substance use 'no=-0.5846+0.0937(pfs)

So the second half of those were probably unnecessary since they are just the inverse of their counterparts in the first two of the models. All p-values < 0.0001. Standard error on either pfs is 0.00542 and on either rfs is 0.00478.

This is to show that, when tested independently, these are behaving as expected: increases in number of risk factors increases the probability of using substances, while increases in protective factors reduces this probability. The problem is arising when they are thrown into a model together.

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

Reeza — Tue, 01 Oct 2019 16:16:05 GMT

This is to show that, when tested independently, these are behaving as expected: increases in number of risk factors increases the probability of using substances, while increases in protective factors reduces this probability. The problem is arising when they are thrown into a model together.

Please show a PROC FREQ of how the variables interact.

As you showed before the proc FREQ returns a p-value of <0.0001 so it seems that they are NOT independent.

Have you tried adding an interaction term?

Re: PROC LOGISTIC: Positive effect in logistic model where a negative one should occur

halkyos — Tue, 01 Oct 2019 17:10:01 GMT

Sorry I saw the request and was having trouble getting it all to show up (13x22 table problems). Hopefully this helps.

I am not familiar with adding interaction terms. Is this just running the model as sub1=rfs pfs rfs*pfs?

(edit: I cropped the images to allow them to be easier to read)