topic Re: Calculating Percentile across observations and excluding missing values in Statistical Procedures

Calculating Percentile across observations and excluding missing values

tts — Tue, 01 Mar 2016 20:11:21 GMT

Hi all,

I'm wondering whether there is a way to calculate percentile across variables for each observation, and excludes missing values in the calculation?

The percentile function seems to work but if there is just a single missing value within all the variables then the result is a missing variable.

Only workaround I was able to find was to achieve this in Excel, as my dataset fortunately wasn't too large.

Thanks for your help!

Re: Calculating Percentile across observations and excluding missing values

Reeza — Tue, 01 Mar 2016 20:19:46 GMT

I would recommend a transpose of your data and then using a more robust procedure such as proc univariate or proc means to calculate the percentile.

However, the behaviour your describing is not consistent with the percentile documentation, which states:

The PCTL function returns the percentile of the nonmissing values corresponding to the percentage. I can't replicate that behaviour as well, can you post a sample of your data where this was occuring?

data test;
array pct(100) pct1-pct100 (1:100);
x=pctl(50, of pct(*));

do i=20 to 30;
pct(i)=.;
end;

y=pctl(50, of pct(*));

keep x y;

run;

Re: Calculating Percentile across observations and excluding missing values

tts — Tue, 01 Mar 2016 22:52:32 GMT

My data is structured similarly to your test dataset, however my variable names are not standardized. There is no standardization as they are various company tickers on NASDAQ.

The way you define the variables to calculate percentile across would not work in my instance, correct?

What I had tried was :

percentile= PCTL(95,A--VLU); with A and VLU being the first and last variables I am interested in calculating percentile across.

Thanks!

Re: Calculating Percentile across observations and excluding missing values

Reeza — Tue, 01 Mar 2016 23:04:27 GMT

If you can define it that way, you could also define your array similarily.

array stocks(*) A--VLU;

At any rate, I still can't replicate your issue. Please post your code and sample data that replicates your problem. I'm guessing you actually have some other issue.

data test;
array pct(100) pct1-pct100 (1:100);
x=pctl(50, of pct(*));

do i=20 to 30;
pct(i)=.;
end;

y=pctl(50, of pct(*));
z=pctl(95, pct1--pct100);

keep x y z;

run;

proc print;run;

Re: Calculating Percentile across observations and excluding missing values

FreelanceReinh — Wed, 02 Mar 2016 09:15:06 GMT

Hi @Reeza,

I think you should insert an "of" into your definition of z. Otherwise, the 95th percentile of a single value (singleton set), pct1--pct100=pct1+pct100=101, will be calculated.

Re: Calculating Percentile across observations and excluding missing values

Reeza — Thu, 03 Mar 2016 02:17:47 GMT

@FreelanceReinh

Thanks!

The corrected code is below and still does not replicate the problem:

data test;
array pct(100) pct1-pct100 (1:100);
x=pctl(50, of pct(*));

do i=20 to 30;
pct(i)=.;
end;

y=pctl(50, of pct(*));
z=pctl(50, of pct1--pct100);
q=pctl(50, pct1, pct2, pct3, pct21, pct25, pct29, pct30, pct50, pct99, pct100);

keep x y z q;

run;

proc print;run;

@tts Have you been able to replicate your issue?