topic Re: Why do the periodic functions not include a median aggregation? in SAS Visual Analytics

Why do the periodic functions not include a median aggregation?

t_rees0498 — Wed, 06 Oct 2021 14:17:42 GMT

For reference, I'm using release 8.5.1 on Viya 3.05. The data I work with is typically in a time series. I've been using the periodic functions to calculate percentage growth rates. These functions are great, but I don't understand how a statistical tool implements period averages and not medians. It's a fundamental statistical concept that these two (average and median) are not equal for any asymmetric distribution, so from my perspective it should be an add. I understand there are backend requirements that may limit this (although the min and max are included amongst others) and you can use code to calculate these, which I currently do, but it seems incredibly inefficient for me to create additional data just to compute median growth rates. Thanks for listening to my rant, and if you have any thoughts/insight into this, that would be great.

Re: Why do the periodic functions not include a median aggregation?

sbxkoenk — Wed, 06 Oct 2021 16:56:18 GMT

Hello,

I think it has to do with the distributed computing which takes place in the back rooms of Visual Analytics (a mean probably being easier than a median in such a "parallel" context).

In SAS/ETS and SAS Econometrics you find MULTPLE ways for median aggregation of time series data.

I guess you are using the periodic functions offered in VA and not some procedures to pre-calculate what you need, correct?

Cheers,

Koen

Re: Why do the periodic functions not include a median aggregation?

t_rees0498 — Wed, 06 Oct 2021 21:01:50 GMT

Hello,

Thanks for responding - the insight helps solve my curiosity, but I can't imagine the computational requirements to be too limiting (especially since we are using VA on Viya with CAS). I am currently using SAS code to create this, so there is definitely a solution in place. For the work I generally do, it just saves me additional time of creating more data just so I can compute median percentage changes over time. This becomes more challenging if I want to map the data for filtering in VA, thus requiring multiple datasets stratified by different categories. Bottom line, if other analysts have this problem, I think this would be a great add to the periodic functions. Is there a place where I can add product recommendations etc.?

Thanks,

Tom

Re: Why do the periodic functions not include a median aggregation?

sbxkoenk — Thu, 07 Oct 2021 09:24:00 GMT

Hello,

Of course SAS VIYA with CAS-engine is extremely powerful and so there's no lack of computational power.

What I meant was : it's easy to calculate a mean when the data (and computations) are distributed across multiple nodes (you take the weighted mean of the means on the nodes). For a median, this is more complicated (but still possible I would think).

I am still hoping a VA-specialist will chime in to shed some more light on this. I am only a basic user of VA. I cannot confirm it is NOT possible in VA.

> Is there a place where I can add product recommendations etc.?
Of course, the SASWare Ballot!
Navigate to the Community header (drop-down list) in the top-left corner. Click on SASWare Ballot and there you go.

Good luck,
Koen

Re: Why do the periodic functions not include a median aggregation?

Kurt_Bremser — Thu, 07 Oct 2021 09:40:07 GMT

A median is not so much the result of a calculation, but more of a location within the data, and the location is completely lost when you split the data. To determine the median, you have to have a complete set of data, sorted, and then select the midpoint by observation number (or the mean between the midpoints when you have an even number).

Re: Why do the periodic functions not include a median aggregation?

t_rees0498 — Fri, 08 Oct 2021 16:33:01 GMT

Thanks for the reply, I suppose this does make sense, although it still is frustrating.

Re: Why do the periodic functions not include a median aggregation?

t_rees0498 — Fri, 08 Oct 2021 16:34:05 GMT

Thanks for the information and your help!