Programming the statistical procedures from SAS

Concordance correlation coefficient

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 11
Accepted Solution

Concordance correlation coefficient

Hi

I have a data set that I have been told I need to perform a concordance correlation coefficient test on. It is a correlation between two measures in a time serie. I have tried to find a totutial here on how to perform this test in SAS but have not been able to find one. I do therefore hope that there is someone on this cummuniti that can help me with this problem.

I have performed a test here https://services.niwa.co.nz/services/statistical/concordance but since this is a web based test where I have no idea of what is going on behind the screen, I would prefere to perform a proper test in SAS.

Kind regards,

Per Nielsen


Accepted Solutions
Solution
‎06-13-2017 08:21 AM
PROC Star
Posts: 7,659

Re: Concordance correlation coefficient

Posted in reply to PerNielsen

You can find an example at: https://onlinecourses.science.psu.edu/stat509/node/161

 

Art, CEO, AnalystFinder.com

 

View solution in original post


All Replies
Solution
‎06-13-2017 08:21 AM
PROC Star
Posts: 7,659

Re: Concordance correlation coefficient

Posted in reply to PerNielsen

You can find an example at: https://onlinecourses.science.psu.edu/stat509/node/161

 

Art, CEO, AnalystFinder.com

 

Super User
Posts: 20,731

Re: Concordance correlation coefficient

I'm not sure Cohens Kappa is the correct test for a time series, my stats is getting rusty. 

First determine the appropriate tests that need to occur and then figure out how to perform the test in SAS. 

Occasional Contributor
Posts: 11

Re: Concordance correlation coefficient

Worked perfectly, got the same results as from the home page, just feels a lot better to have a SAS program doing the analysis. Thanks for this suggestion!

Occasional Contributor
Posts: 11

Re: Concordance correlation coefficient

Hi Art,

 

I hope it is okay that I email you with an additional question to this one.

I have been asked to include repeated measures in my analysis, but have been discussing this with a statistician who on the other hand suggested that I did a CCC analysis for each animal (I have 20) and then made an additional coloum to the table with results for min-max for these 20 CCCs'.

Normally when you run a Proc statement in SAS you can include a BY statement, do you know if this is possible in a PROC IMLas this one:

 

proc iml;
use data.ice_cs;
read all var {iceMinlying} into var1;
read all var {csMinlying} into var2;
start concorr;
nonmiss=loc(var1#var2^=.);
var1=var1[nonmiss];
var2=var2[nonmiss];
free nonmiss;
n=nrow(var1);
mu1=sum(var1)/n;
mu1=round(mu1,0.0001);
mu2=sum(var2)/n;
mu2=round(mu2,0.0001);
sigma11=ssq(var1-mu1)/(n-1);
sigma11=round(sigma11,0.0001);
sigma22=ssq(var2-mu2)/(n-1);
sigma22=round(sigma22,0.0001);
sigma12=sum((var1-mu1)#(var2-mu2))/(n-1);
sigma12=round(sigma12,0.0001);
lshift=(mu1-mu2)/((sigma11#sigma22)##0.25);
rho=sigma12/sqrt(sigma11#sigma22);
rho=round(rho,0.0001);
z=log((1+rho)/(1-rho))/2;
se_z=1/sqrt(n-3);
t=tinv(0.975,n-3);
z_low=z-(se_z#t);
z_upp=z+(se_z#t);
rho_low=(exp(2#z_low)-1)/(exp(2#z_low)+1);
rho_low=round(rho_low,0.0001);
rho_upp=(exp(2#z_upp)-1)/(exp(2#z_upp)+1);
rho_upp=round(rho_upp,0.0001);
crho=(2#sigma12)/((sigma11+sigma22)+((mu1-mu2)##2));
crho=round(crho,0.0001);
z=log((1+crho)/(1-crho))/2;
if sigma12^=0 then do;
   t1=((1-(rho##2))#(crho##2))/((1-(crho##2))#(rho##2));
   t2=(2#(crho##3)#(1-crho)#(lshift##2))/(rho#((1-(crho##2))##2));
   t3=((crho##4)#(lshift##4))/(2#(rho##2)#((1-(crho##2))##2));
   se_z=sqrt((t1+t2-t3)/(n-2));
end;
else se_z=sqrt(2#sigma11#sigma22)/((sigma11+sigma22+((mu1-mu2)##2))#(n-2));
t=tinv(0.975,n-2);
z_low=z-(se_z#t);
z_upp=z+(se_z#t);
crho_low=(exp(2#z_low)-1)/(exp(2#z_low)+1);
crho_low=round(crho_low,0.0001);
crho_upp=(exp(2#z_upp)-1)/(exp(2#z_upp)+1);
crho_upp=round(crho_upp,0.0001);
Results=n//mu1//mu2//sigma11//sigma22//sigma12//rho_low//rho//rho_upp//
        crho_low//crho//crho_upp;
r_name={'SampleSize' 'Mean_1' 'Mean_2' 'Variance_1' 'Variance_2' 'Covariance' 'Corr LowerCL'
       'Corr' 'Corr UpperCL' 'ConcCorr LowerCL' 'ConcCorr' 'ConcCorr UpperCL'};
print 'The Estimated Correlation and Concordance Correlation (and 95% Confidence Limits) IceTag vs CowScout lying';
print Results [rowname=r_name];
finish concorr;
run concorr;

 

Kind regards,

Per

PROC Star
Posts: 7,659

Re: Concordance correlation coefficient

Posted in reply to PerNielsen

Per: Unfortunately, I've never learned IML. However, I'm sure that @Rick_SAS@Ksharp or others (who are) could easily answer your question.

 

That said, worse case, since the IML use statement accepts dataset options, you could always wrap the code in a macro that uses a %do loop, and supply the by condition in a where=() dataset option.

 

Art, CEO, AnalystFinder.com

 

SAS Super FREQ
Posts: 3,839

Re: Concordance correlation coefficient

@art297 wrote:

That said, worse case, since the IML use statement accepts dataset options, you could always wrap the code in a macro that uses a %do loop, and supply the by condition in a where=() dataset option.

No. You almost never use a %DO loop in IML programs. SAS/IML contains DO loops, IF-THEN statements, etc, as part of the language, so you can perform loops and conditional logic within the language.

 

Although SAS/IML does not support a BY statement, there are many ways to perform BY-group processing:

Occasional Contributor
Posts: 11

Re: Concordance correlation coefficient

Thanks for your swift responses.

I assume the UNIQUE-LOC is the procedure that might work on my data and I will test it with my limited knowledge of programming out of my comfort zone in SAS.

Otherwise I just have to perform 300 runs of the procedure :-) I will specify the animal number, out of 30, that I have in my dataset and then run the analysis. Would however have been nice to have a procedure in sas that does this for me in the proc iml, but I understand that it is not that simple :-)

 

Kind regards,

Per

PROC Star
Posts: 7,659

Re: Concordance correlation coefficient

@Rick_SAS: Thanks for providing the info. However, I do have to ask, 'Why are macros almost never used' in IML?

 

Since, like @PerNielsen, I don't know anything about IML, if I was asked to take the IML code you provided in one of your loop blogs and run the code separately for each gender, I'd do something like the macro wrapper shown below. I created a numeric field called gender simply because @PerNielsen mentioned that he had to repeat the code for animals 1 thru 30.

 

Is there a problem doing it this way, or is it just preferred to take advantage of the IML functionality?

 

Art, CEO, AnalystFinder.com

data testdata;
  set sashelp.class;
  if sex eq 'M' then gender=1;
  else gender=2;
run;

%macro loopiml;
%do i=1 %to 2;
title "gender eq &i";
proc iml;
use testdata (where=(gender eq &i.)); 
   read all var {age height} into m;
close;
call sort(m, 1);         /* 1. Sort data by first column */
C = m[,1];               /* finished with m: extract sorted categories */
x = m[,2];               /* and sorted data */

/* 2. Obtain row numbers for the first observation in each level. */
b = uniqueby(C, 1);     /* b[i] = beginning of i_th category */
u = C[b];               /* get unique values (if needed) */
print b u;

s = j(nrow(b),1);         /* 3. Allocate vector to hold results */
b = b // (nrow(C)+1);     /* trick: append (n+1) to end of b */
do i = 1 to nrow(b)-1;    /* 4. For each level... */
   idx = b[i]:(b[i+1]-1); /* 5. Find observations in level */
   s[i] = mean(x[idx]);   /* 6. Compute statistic on those values */
end;
lbl = putn(u, "Best4.");  /* convert numeric values to character */
print s[rowname=lbl];
quit;
  %end;
%mend loopiml;
%loopiml;
SAS Super FREQ
Posts: 3,839

Re: Concordance correlation coefficient

I do have to ask, 'Why are macros almost never used' in IML?

 

Because macro replication is unnecessary and is inefficient compared to native IML processing. In a macro, you start/stop the procedure &N times, open/close/read the data set &N times, and allocate/free all the matrices &N times. Using native IML code you start/stop the procedure once, open/close/read the data once, and can make more efficient use of memory. A lot of these reasons are the same reasons that BY-processing is preferred over macro loops when performing computationally intensive tasks such as simulation and bootstrapping.

Super User
Posts: 10,213

Re: Concordance correlation coefficient

Arthur.T ,

In IML , you can define a function which can generate the code dynamically just as MACRO did .

SAS Super FREQ
Posts: 3,839

Re: Concordance correlation coefficient

I think @Ksharp is making a good point about reusability. A macro loop generates multiple copies of code. Each copy must be parsed. In contrast, if you define an IML function (or block of code) and call it in a loop, the code is parsed once but executed multiple times with different arguments. 

Super User
Posts: 20,731

Re: Concordance correlation coefficient

Posted in reply to PerNielsen

Look at PROC TIMESERIES and the OUTCORR option that looks at auto correlation and partial correlations. 

PROC ARIMA / AUTOREG should also be helpful. 

Occasional Contributor
Posts: 11

Re: Concordance correlation coefficient

Thanks Reeza for your suggestions. I have had this through a statistician, since I am also a bit rusty on that point, and they suggested this analysis

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 13 replies
  • 479 views
  • 1 like
  • 5 in conversation