topic Re: WEIGHT calculation in SAS Procedures

WEIGHT calculation

Ankur — Wed, 27 Nov 2013 06:28:19 GMT

Hi,

In order to measure performance and whether we can reduce any execution time, I am trying to convert some of SAS code into SQL query. While doing it, I came across a piece of code weight is being calculated. Need to know how SAS calculates weight for any given column so that I can try and replicate the same using SQL.

proc means data=input_file noprint;

var C_A/weight=T_A;

by period;

output out=output_file

mean=

Any insight on this would be of great help to me.

Thanks,

ankur

Re: WEIGHT calculation

Patrick — Wed, 27 Nov 2013 08:51:45 GMT

The formulas used are - if not obvious - normally documented as part of the documentation for the procedure.

Not sure if changing everything to SQL will help you a lot unless the data tables are stored in a data base. If stored in a data base: You might want to investigate the in-database capabilities of some of the PROCs (eg. Proc Means).

The quickest way of improving performance is in my experience to reduce passes through the data, reduction of volumes as fast as possible, re-design lookups (eg. by using hash tables).

Re: WEIGHT calculation

Ankur — Wed, 27 Nov 2013 10:00:28 GMT

Patrick, Appreciate your response and I do agree with the Performance improvement point.

But as per the assignment, I need some way to do it in SQL. If some one could provide a sample input/output with the weight function, then I can re-engineer it to see how its calculated.

Thanks & Best Regards,

Ankur Vatsal

Re: WEIGHT calculation

MarkFisher — Thu, 28 Nov 2013 15:33:24 GMT

It would take two passes through the data to get the same result in SQL, but the final formula would be:

C_A*weight/sum_of_weights as weighted_mean

where the sum of weights would need to be calculated for each period. You would need to join the summed weights back to the original data.

Re: WEIGHT calculation

MarkFisher — Thu, 28 Nov 2013 15:36:04 GMT

I clicked too soon--the weighted mean for each period would be the sum of C_A*weight divided by the sum of the weights, for each period.

Re: WEIGHT calculation

Reeza — Thu, 28 Nov 2013 15:54:11 GMT

It's a weighted average

Weighted arithmetic mean - Wikipedia, the free encyclopedia

Additionally you need to account for missing and negative weights separately

Weight value...	PROC MEANS...
0	counts the observation in the total number of observations
less than 0	converts the value to zero and counts the observation in the total number of observations
missing	excludes the observation