Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Home
- /
- Programming
- /
- SAS Procedures
- /
- proc means standard deviation too high? SAS 9.4

- RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

🔒 This topic is **solved** and **locked**.
Need further help from the community? Please
sign in and ask a **new** question.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Posted 06-14-2018 11:25 PM
(1808 views)

When I run proc means, the standard deviations are very high.

The variables in the statement below are all binary, so the std should be sqrt(p*(1-p)), as shown in the table below. Instead they are orders of magnitude bigger. I get the same problem with continuous variables; stata delivers much smaller std errors.

No-one else seems to have this problem, and Ive been having it for years, so it has to be something I have misunderstood about proc means.I t might have something to do with the weight statement. The weights post_wt range in size from 350 to 33500.

Please let me know how to get the correct std dev. I'm getting tired of porting my sas data sets to stata.

THanks

J

Variable | N | Mean | Std Dev | Minimum | Maximum | sqrt(p(1-p)) |

PregMonth | 296527 | 0.006651 | 5.973354 | 0 | 1 | 0.081281 |

PregBirth | 296527 | 0.003967 | 4.619279 | 0 | 1 | 0.062856 |

MarrMonth | 296527 | 0.006756 | 6.020052 | 0 | 1 | 0.081917 |

ShotGunMar | 296527 | 0.00106 | 2.391512 | 0 | 1 | 0.032542 |

20462 proc means data=sinwom; *noprint;;

20463 var PregMonth PregBirth MarrMonth ShotGunMar;

20464 *output out=temp mean = PregMonth PregBirth MarrMonth ShotGunMar;

20465 weight post_wt;

20466 *proc print;

20467 run;

NOTE: There were 296527 observations read from the data set WORK.SINWOM.

NOTE: PROCEDURE MEANS used (Total process time):

real time 0.19 seconds

cpu time 0.29 seconds

1 ACCEPTED SOLUTION

Accepted Solutions

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

```
proc means data=sinwom vardef=wdf ;
var PregMonth PregBirth MarrMonth ShotGunMar;
weight post_wt;
run;
```

-------------------

Edit: I am sorry that my code is wrong, because i missed your below comment.

>The variables in the statement below are all binary,

6 REPLIES 6

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

```
proc means data=sinwom vardef=wdf ;
var PregMonth PregBirth MarrMonth ShotGunMar;
weight post_wt;
run;
```

-------------------

Edit: I am sorry that my code is wrong, because i missed your below comment.

>The variables in the statement below are all binary,

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Formula is sqrt(n*p*(1-p)) isn't it?

Have you tried using PROC SURVEYMEANS if you have weighed data, from surveys in particular, it may be more appropriate.

There's also a note in PROC MEANS weight statement:

Single extreme weight values can cause inaccurate results.

When one (and only one) weight value is many orders of magnitude larger than the other weight values (for example, 49 weight values of 1 and one weight value of 1×10^{14}), certain statistics might not be within acceptable accuracy limits. The affected statistics are based on the second moment (such as standard deviation, corrected sum of squares, variance, and standard error of the mean). Under certain circumstances, no warning is written to the SAS log.

The detailed section on weights has more information that may help you narrow down the issue.

Can you replicate these results on someone else's machine? Or provide a sample that reflects this issue. You don't have to provide any individual information, just the 1/0 and the respective weight so we can replicate the issue and debug if necessary.

Edit: From @amatsu users answer, the weight pages has examples. In their case, the weights increase the StDev though.

@john_knowles wrote:

When I run proc means, the standard deviations are very high.

The variables in the statement below are all binary, so the std should be sqrt(p*(1-p)), as shown in the table below. Instead they are orders of magnitude bigger. I get the same problem with continuous variables; stata delivers much smaller std errors.

No-one else seems to have this problem, and Ive been having it for years, so it has to be something I have misunderstood about proc means.I t might have something to do with the weight statement. The weights post_wt range in size from 350 to 33500.

Please let me know how to get the correct std dev. I'm getting tired of porting my sas data sets to stata.

THanks

J

Variable N Mean Std Dev Minimum Maximum sqrt(p(1-p)) PregMonth 296527 0.006651 5.973354 0 1 0.081281 PregBirth 296527 0.003967 4.619279 0 1 0.062856 MarrMonth 296527 0.006756 6.020052 0 1 0.081917 ShotGunMar 296527 0.00106 2.391512 0 1 0.032542 20462 proc means data=sinwom; *noprint;;

20463 var PregMonth PregBirth MarrMonth ShotGunMar;

20464 *output out=temp mean = PregMonth PregBirth MarrMonth ShotGunMar;

20465 weight post_wt;

20466 *proc print;

20467 run;

NOTE: There were 296527 observations read from the data set WORK.SINWOM.

NOTE: PROCEDURE MEANS used (Total process time):

real time 0.19 seconds

cpu time 0.29 seconds

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Be aware that weights are not frequencies. If these numbers represent the number of cases, you should use the FREQ statement instead of the WEIGHT statement.

If you really have weights, then the SAS documentation provides the formulas for weighted statistics. Compare those formulas against the formulas for the other software. My guess is that it is a divisor issue. SAS provides four possible divisors for a weighted statistic such as the StdDev. By default, the divisor is (n-1), which is the usual unweighted divisor. Try using VARDEF=WEIGHT on the PROC MEANS statement to use the sum of weights as the divisor.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.

**If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website. **

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.