topic Re: What happens if I don't use person weights and/or sampling weights? in Statistical Procedures

What happens if I don't use person weights and/or sampling weights?

Wolverine — Mon, 20 May 2024 12:14:08 GMT

As I detailed in this post I'm trying to make a "Table 1" summary table and another additional table of descriptive statistics for dozens of variables. Generating the proper results is relatively easy. But rearranging those results for dozens of variables and putting them into a well-organized summary table takes a lot of code. The TableN macro does this extremely well, but it doesn't have a provision to add weights. Likewise, the code I wrote for the secondary table uses Proc Ttest, and there is no "SURVEY" equivalent (like Proc Surveyfreq is the equivalent to Proc Freq). Proc Ttest can use a person weight, but not sampling weights.

I'm supposed to have a meeting this afternoon to review the results. At this point, we just need initial results to start developing a theoretical model. So what kind of error will these results have? As I understand it, the standard error will be underestimated if weights are not used. As a result, some findings may appear to be significant but really aren't. Is that correct, and are there any other issues I should be aware of ?

Re: What happens if I don't use person weights and/or sampling weights?

PaigeMiller — Mon, 20 May 2024 14:07:50 GMT

If you don't use the person and replicate weights, it's not just the standard errors that are wrong, but the estimates are wrong as well.

The survey equivalent of PROC TTEST:

https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.4/statug/statug_surveymeans_examples06.htm

https://communities.sas.com/t5/Statistical-Procedures/How-I-use-t-test-in-complex-survey-deign/td-p/427092

Re: What happens if I don't use person weights and/or sampling weights?

ballardw — Mon, 20 May 2024 20:42:26 GMT

If the data set you have requires weights to analyze / report properly and a macro or other tool does not support weights then the basic approach would be to use one of the procedures to generate the POPULATION estimates to have the counts and rates with confidence limits. Then use the report tool with the new data.

IF that tool wants to calculate confidence intervals then hope there is a way to override that behavior as it may have problems with the new data set and calculating incorrect confidence limits.

@Wolverine wrote:

As I detailed in this post I'm trying to make a "Table 1" summary table and another additional table of descriptive statistics for dozens of variables. Generating the proper results is relatively easy. But rearranging those results for dozens of variables and putting them into a well-organized summary table takes a lot of code. The TableN macro does this extremely well, but it doesn't have a provision to add weights. Likewise, the code I wrote for the secondary table uses Proc Ttest, and there is no "SURVEY" equivalent (like Proc Surveyfreq is the equivalent to Proc Freq). Proc Ttest can use a person weight, but not sampling weights.

I'm supposed to have a meeting this afternoon to review the results. At this point, we just need initial results to start developing a theoretical model. So what kind of error will these results have? As I understand it, the standard error will be underestimated if weights are not used. As a result, some findings may appear to be significant but really aren't. Is that correct, and are there any other issues I should be aware of ?