turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- Analytics
- /
- Stat Procs
- /
- Concordance correlation coefficient

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

06-11-2017 02:29 PM

Hi

I have a data set that I have been told I need to perform a concordance correlation coefficient test on. It is a correlation between two measures in a time serie. I have tried to find a totutial here on how to perform this test in SAS but have not been able to find one. I do therefore hope that there is someone on this cummuniti that can help me with this problem.

I have performed a test here https://services.niwa.co.nz/services/statistical/concordance but since this is a web based test where I have no idea of what is going on behind the screen, I would prefere to perform a proper test in SAS.

Kind regards,

Per Nielsen

Accepted Solutions

Solution

06-13-2017
08:21 AM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to PerNielsen

06-11-2017 02:43 PM

You can find an example at: https://onlinecourses.science.psu.edu/stat509/node/161

Art, CEO, AnalystFinder.com

All Replies

Solution

06-13-2017
08:21 AM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to PerNielsen

06-11-2017 02:43 PM

You can find an example at: https://onlinecourses.science.psu.edu/stat509/node/161

Art, CEO, AnalystFinder.com

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to art297

06-11-2017 02:53 PM

I'm not sure Cohens Kappa is the correct test for a time series, my stats is getting rusty.

First determine the appropriate tests that need to occur and then figure out how to perform the test in SAS.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to art297

06-13-2017 08:24 AM

Worked perfectly, got the same results as from the home page, just feels a lot better to have a SAS program doing the analysis. Thanks for this suggestion!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to art297

Wednesday

Hi Art,

I hope it is okay that I email you with an additional question to this one.

I have been asked to include repeated measures in my analysis, but have been discussing this with a statistician who on the other hand suggested that I did a CCC analysis for each animal (I have 20) and then made an additional coloum to the table with results for min-max for these 20 CCCs'.

Normally when you run a Proc statement in SAS you can include a BY statement, do you know if this is possible in a PROC IMLas this one:

proc iml;

use data.ice_cs;

read all var {iceMinlying} into var1;

read all var {csMinlying} into var2;

start concorr;

nonmiss=loc(var1#var2^=.);

var1=var1[nonmiss];

var2=var2[nonmiss];

free nonmiss;

n=nrow(var1);

mu1=sum(var1)/n;

mu1=round(mu1,0.0001);

mu2=sum(var2)/n;

mu2=round(mu2,0.0001);

sigma11=ssq(var1-mu1)/(n-1);

sigma11=round(sigma11,0.0001);

sigma22=ssq(var2-mu2)/(n-1);

sigma22=round(sigma22,0.0001);

sigma12=sum((var1-mu1)#(var2-mu2))/(n-1);

sigma12=round(sigma12,0.0001);

lshift=(mu1-mu2)/((sigma11#sigma22)##0.25);

rho=sigma12/sqrt(sigma11#sigma22);

rho=round(rho,0.0001);

z=log((1+rho)/(1-rho))/2;

se_z=1/sqrt(n-3);

t=tinv(0.975,n-3);

z_low=z-(se_z#t);

z_upp=z+(se_z#t);

rho_low=(exp(2#z_low)-1)/(exp(2#z_low)+1);

rho_low=round(rho_low,0.0001);

rho_upp=(exp(2#z_upp)-1)/(exp(2#z_upp)+1);

rho_upp=round(rho_upp,0.0001);

crho=(2#sigma12)/((sigma11+sigma22)+((mu1-mu2)##2));

crho=round(crho,0.0001);

z=log((1+crho)/(1-crho))/2;

if sigma12^=0 then do;

t1=((1-(rho##2))#(crho##2))/((1-(crho##2))#(rho##2));

t2=(2#(crho##3)#(1-crho)#(lshift##2))/(rho#((1-(crho##2))##2));

t3=((crho##4)#(lshift##4))/(2#(rho##2)#((1-(crho##2))##2));

se_z=sqrt((t1+t2-t3)/(n-2));

end;

else se_z=sqrt(2#sigma11#sigma22)/((sigma11+sigma22+((mu1-mu2)##2))#(n-2));

t=tinv(0.975,n-2);

z_low=z-(se_z#t);

z_upp=z+(se_z#t);

crho_low=(exp(2#z_low)-1)/(exp(2#z_low)+1);

crho_low=round(crho_low,0.0001);

crho_upp=(exp(2#z_upp)-1)/(exp(2#z_upp)+1);

crho_upp=round(crho_upp,0.0001);

Results=n//mu1//mu2//sigma11//sigma22//sigma12//rho_low//rho//rho_upp//

crho_low//crho//crho_upp;

r_name={'SampleSize' 'Mean_1' 'Mean_2' 'Variance_1' 'Variance_2' 'Covariance' 'Corr LowerCL'

'Corr' 'Corr UpperCL' 'ConcCorr LowerCL' 'ConcCorr' 'ConcCorr UpperCL'};

print 'The Estimated Correlation and Concordance Correlation (and 95% Confidence Limits) IceTag vs CowScout lying';

print Results [rowname=r_name];

finish concorr;

run concorr;

Kind regards,

Per

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to PerNielsen

Wednesday

Per: Unfortunately, I've never learned IML. However, I'm sure that @Rick_SAS, @Ksharp or others (who are) could easily answer your question.

That said, worse case, since the IML use statement accepts dataset options, you could always wrap the code in a macro that uses a %do loop, and supply the by condition in a where=() dataset option.

Art, CEO, AnalystFinder.com

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to art297

Wednesday

@art297 wrote:

That said, worse case, since the IML use statement accepts dataset options, you could always wrap the code in a macro that uses a %do loop, and supply the by condition in a where=() dataset option.

No. You almost never use a %DO loop in IML programs. SAS/IML contains DO loops, IF-THEN statements, etc, as part of the language, so you can perform loops and conditional logic within the language.

Although SAS/IML does not support a BY statement, there are many ways to perform BY-group processing:

- The UNIQUE-LOC technique
- The UNIQUEBY technique
- Use the WHERE clause within a loop to read subsets of the data (Not as efficient, but less bookkeeping. Also you can use it for large data sets that do not fit into RAM.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to Rick_SAS

Wednesday

Thanks for your swift responses.

I assume the UNIQUE-LOC is the procedure that might work on my data and I will test it with my limited knowledge of programming out of my comfort zone in SAS.

Otherwise I just have to perform 300 runs of the procedure :-) I will specify the animal number, out of 30, that I have in my dataset and then run the analysis. Would however have been nice to have a procedure in sas that does this for me in the proc iml, but I understand that it is not that simple :-)

Kind regards,

Per

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to Rick_SAS

Wednesday

@Rick_SAS: Thanks for providing the info. However, I do have to ask, 'Why are macros almost never used' in IML?

Since, like @PerNielsen, I don't know anything about IML, if I was asked to take the IML code you provided in one of your loop blogs and run the code separately for each gender, I'd do something like the macro wrapper shown below. I created a numeric field called gender simply because @PerNielsen mentioned that he had to repeat the code for animals 1 thru 30.

Is there a problem doing it this way, or is it just preferred to take advantage of the IML functionality?

Art, CEO, AnalystFinder.com

data testdata; set sashelp.class; if sex eq 'M' then gender=1; else gender=2; run; %macro loopiml; %do i=1 %to 2; title "gender eq &i"; proc iml; use testdata (where=(gender eq &i.)); read all var {age height} into m; close; call sort(m, 1); /* 1. Sort data by first column */ C = m[,1]; /* finished with m: extract sorted categories */ x = m[,2]; /* and sorted data */ /* 2. Obtain row numbers for the first observation in each level. */ b = uniqueby(C, 1); /* b[i] = beginning of i_th category */ u = C[b]; /* get unique values (if needed) */ print b u; s = j(nrow(b),1); /* 3. Allocate vector to hold results */ b = b // (nrow(C)+1); /* trick: append (n+1) to end of b */ do i = 1 to nrow(b)-1; /* 4. For each level... */ idx = b[i]:(b[i+1]-1); /* 5. Find observations in level */ s[i] = mean(x[idx]); /* 6. Compute statistic on those values */ end; lbl = putn(u, "Best4."); /* convert numeric values to character */ print s[rowname=lbl]; quit; %end; %mend loopiml; %loopiml;

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to art297

Wednesday

> * I do have to ask, 'Why are macros almost never used' in IML?*

Because macro replication is unnecessary and is inefficient compared to native IML processing. In a macro, you start/stop the procedure &N times, open/close/read the data set &N times, and allocate/free all the matrices &N times. Using native IML code you start/stop the procedure once, open/close/read the data once, and can make more efficient use of memory. A lot of these reasons are the same reasons that BY-processing is preferred over macro loops when performing computationally intensive tasks such as simulation and bootstrapping.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to art297

Thursday

Arthur.T ,

In IML , you can define a function which can generate the code dynamically just as MACRO did .

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to Ksharp

Thursday

I think @Ksharp is making a good point about reusability. A macro loop generates multiple copies of code. Each copy must be parsed. In contrast, if you define an IML function (or block of code) and call it in a loop, the code is parsed once but executed multiple times with different arguments.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to PerNielsen

06-11-2017 02:51 PM

Look at PROC TIMESERIES and the OUTCORR option that looks at auto correlation and partial correlations.

PROC ARIMA / AUTOREG should also be helpful.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to Reeza

06-13-2017 08:22 AM

Thanks Reeza for your suggestions. I have had this through a statistician, since I am also a bit rusty on that point, and they suggested this analysis