PROC SURVEYFREQ note in log - "...the design correction is not positive"

[wfung]

I'm running PROC SURVEYFREQ to compare demographic covariables of my sample with a national sample.

The first step I wanted to do was determine if propensity score weighting can balance the demographics covariates between my sample and the national sample. When I ran the PROC SURVEYFREQ with chi-square test, I got the following note in the SAS log: 

"The Rao-Scott chi-square test cannot be computed for the table of GENDER by Treatment
because the design correction is not positive."

I got the same note for some (but not all) of the covarites. Does anyone know what this means? Any help would be much appreciated. Thank you. 

I first appended my sample and the national sample into one dataset.

Then I created the propensity scores using a logistic regression. 

Here is the syntax that created the propensity score weights: 

/*Calculate weights using propensity score. */
DATA PropensityVol2;
SET PropensityVol;

/*Weighting by the odds*/
IF Treatment = 1 THEN PSWeightWBO = 1;
ELSE IF Treatment = 0 THEN PSWeightWBO = prob/(1 - prob);

/*Inverse probability of treatment*/
IF Treatment = 1 THEN PSWeightIPTW = 1/prob;
ELSE IF Treatment = 0 THEN PSWeightIPTW = 1/(1 - prob);

/*Multiply propensity score weight by survey weight*/
IF Treatment = 1 THEN PSWeightWBOxSurveyWeight = PSWeightWBO * 1;    /*Multiply by 1 because my sample does not have survey weights*/
ELSE IF Treatment = 0 THEN PSWeightWBOxSurveyWeight = PSWeightWBO * NWGTR;    /*Multiply by national sample's survey weight*/

IF Treatment = 1 THEN PSWeightIPTWxSurveyWeight = PSWeightIPTW * 1;
ELSE IF Treatment = 0 THEN PSWeightIPTWxSurveyWeight = PSWeightIPTW * NWGTR;

RUN;

/*Use PSWeightWBOxSurveyWeight or PSWeightIPTWxSurveyWeight  in PROC SURVEYFREQ*/ 

[ballardw]

And the surveyfreq code?

[wfung]

Here is the PROC SURVEYFREQ syntax: 

PROC SURVEYFREQ DATA = PropensityVol2;
TABLES
  (Age_category
  Gender
  Race
  Educ
  Marital
  Income_category
  Veteran
  Employment
  FuncLimit) * Treatment /CHISQ;
WEIGHT PSWeightIPTWxSurveyWeight;

RUN; 

[SAS_Rob · Accepted solution]

The design correction can in fact be negative. I suggest that you take a look at the reference listed below for details on this.

Thomas, D.R., and Rao, J.N.K. (1984), "A Monte Carlo Study of Exact Levels of Goodness-of-Fit Statistics Under Cluster Sampling," Proceedings of the Survey Research Methods Section, ASA, 207 - 211.

As to the reason why in your case it is negative it is hard to say since I am not familiar with your data. It may be related to the fact that you have many single observation strata, but this is not always the cause of it.

I would suggest using the modified Rao-Scott chi-square (option CHISQ1) in this case. It computes the design correction using the null hypothesis cell proportions (instead of the estimated proportions) together with the cell design effects.