topic Re: PROC SURVEYFREQ note in log - "...the design correction is not positive" in Statistical Procedures

PROC SURVEYFREQ note in log - "...the design correction is not positive"

wfung — Mon, 14 Aug 2017 14:55:26 GMT

I'm running PROC SURVEYFREQ to compare demographic covariables of my sample with a national sample.

The first step I wanted to do was determine if propensity score weighting can balance the demographics covariates between my sample and the national sample. When I ran the PROC SURVEYFREQ with chi-square test, I got the following note in the SAS log:

"The Rao-Scott chi-square test cannot be computed for the table of GENDER by Treatment
because the design correction is not positive."

I got the same note for some (but not all) of the covarites. Does anyone know what this means? Any help would be much appreciated. Thank you.

I first appended my sample and the national sample into one dataset.

Then I created the propensity scores using a logistic regression.

Here is the syntax that created the propensity score weights:

/*Calculate weights using propensity score. */
DATA PropensityVol2;
SET PropensityVol;

/*Weighting by the odds*/
IF Treatment = 1 THEN PSWeightWBO = 1;
ELSE IF Treatment = 0 THEN PSWeightWBO = prob/(1 - prob);

/*Inverse probability of treatment*/
IF Treatment = 1 THEN PSWeightIPTW = 1/prob;
ELSE IF Treatment = 0 THEN PSWeightIPTW = 1/(1 - prob);

/*Multiply propensity score weight by survey weight*/
IF Treatment = 1 THEN PSWeightWBOxSurveyWeight = PSWeightWBO * 1; /*Multiply by 1 because my sample does not have survey weights*/
ELSE IF Treatment = 0 THEN PSWeightWBOxSurveyWeight = PSWeightWBO * NWGTR; /*Multiply by national sample's survey weight*/

IF Treatment = 1 THEN PSWeightIPTWxSurveyWeight = PSWeightIPTW * 1;
ELSE IF Treatment = 0 THEN PSWeightIPTWxSurveyWeight = PSWeightIPTW * NWGTR;

RUN;

/*Use PSWeightWBOxSurveyWeight or PSWeightIPTWxSurveyWeight in PROC SURVEYFREQ*/

Re: PROC SURVEYFREQ note in log - "...the design correction is not positive"

ballardw — Mon, 14 Aug 2017 16:20:28 GMT

And the surveyfreq code?

Re: PROC SURVEYFREQ note in log - "...the design correction is not positive"

wfung — Mon, 14 Aug 2017 16:24:41 GMT

Here is the PROC SURVEYFREQ syntax:

PROC SURVEYFREQ DATA = PropensityVol2;
TABLES
(Age_category
Gender
Race
Educ
Marital
Income_category
Veteran
Employment
FuncLimit) * Treatment /CHISQ;
WEIGHT PSWeightIPTWxSurveyWeight;

RUN;

Re: PROC SURVEYFREQ note in log - "...the design correction is not positive"

SAS_Rob — Mon, 14 Aug 2017 17:56:50 GMT

The design correction can in fact be negative. I suggest that you take a look at the reference listed below for details on this.

Thomas, D.R., and Rao, J.N.K. (1984), "A Monte Carlo Study of Exact Levels of Goodness-of-Fit Statistics Under Cluster Sampling," Proceedings of the Survey Research Methods Section, ASA, 207 - 211.

As to the reason why in your case it is negative it is hard to say since I am not familiar with your data. It may be related to the fact that you have many single observation strata, but this is not always the cause of it.

I would suggest using the modified Rao-Scott chi-square (option CHISQ1) in this case. It computes the design correction using the null hypothesis cell proportions (instead of the estimated proportions) together with the cell design effects.