turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- SAS Programming
- /
- SAS Procedures
- /
- how to compare significance (p-value) for diagnost...

Topic Options

- RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

09-16-2016 12:53 PM

Hi all,

I am having issues with finding a right test for comparing accuracy trends over time. Is Cochran-Armitage Trend test is a right choice? I have a 2x2 table comparing test results with gold standard at different time points. I plotted the accuracy trends over time on a graph, but I need to know if there is a significant trend. My data is not on a excel sheet or on SAS. I had to manually calculate accuracy for each time point and then enter them on a excel sheet to generate graphs. In the 2x2 table, If I have empty cells, I added 0.5 to each cell and then calculated accuracy. Any advise is appreciated for calculating and comparing accuracy significance across time. I am attaching the trend grap on here.

My questions are:

1. How to enter data in SAS from a manual 2x2 table, especially when I have 2x2 table data at 4 different time points and I have to do accuracy calculations for 6 different conditions?

2. How to calcualte accuracy using SAS (and how to adjust missing cells)?

3. How to compare accuracy trends (significance, p-value) across 4 different time points?

4. My sample size is low as it is a piolt study involving 18 students.

Thank you for the help,

Sat

- Mark as New
- Bookmark
- Subscribe
- RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to smunigala

09-16-2016 02:23 PM

Can you show us what the 'gold standard' data you are using for comparison looks like?

And describe the manual calculations you did?.

In a generic sense your data need would likely be 4 data columns: the time point value, an idicator to show which column,which row and a value.

Suppose the 2X2 looks something like:

Col1 col

row1 10 15

row2 17 33

Then the data would look like

Time Row Col Value

10 1 1 10

10 2 1 17

10 1 2 15

10 2 2 33

If you want to have multple topics then add a topic or condition variable.

The actual order would typically not be import as Proc Sort can fix if needed.

To get as SAS you could enter into the editor as:

```
data want;
input Time Row Col Value;
datalines;
10 1 1 10
10 2 1 17
10 1 2 15
10 2 2 33
;
run;
```

If you are careful, use Excel to enter the data as above, save as a CSV and use proc import to read into a SAS data set.

It is possible to direct entry in SAS but is often cumbersome.

Suppose your Col1 and Col2 represent Male Female.

Then we ineffect are saying that a value of 1 for column indicates Male and 2 indicates Female.

A format could be used to get pretty text in that case.

You may have to describe what you intend by "adjust missing cells". Do you mean to impute a missing value or exclude from analysis.

Trends are often evaluated by significance of the slope of a line, or significance of parameters in a non-linear, regression model. But 4 points may not be very sensitive.

- Mark as New
- Bookmark
- Subscribe
- RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to ballardw

09-16-2016 02:52 PM

Hi Ballard,

Thanks for the reply. I created the excel sheet with the required data. The data is about how trainee fellows report their findings compared to expert (gold standard). So I have these values for several variables (Variable). I need to find accuracy for different time points (baseline, 5, 10, 15, 20- highlighted in RED). I want to know if there is any significant different in accuracy% between different time points (baseline, 5, 10, 15, 20).

I also highligthed a set of 4 values, where there were empty cells on 2x2 table (missing counts). I added 0.5 to each cell.

Thanks for your help!

Sat

- Mark as New
- Bookmark
- Subscribe
- RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to smunigala

09-16-2016 06:12 PM

Graphically, you could look at the evolution of agreement statistics over time

```
libname xl Excel "&sasforum\datasets\Data_2_by_2.xlsx" access=readonly;
proc sql;
create table two as
select coalesce(study, 0) as study,
variable, fellow, Gold_standard,
int(count) as n
from xl.'Sheet1$'n
order by variable, study;
quit;
libname xl clear;
proc freq data=two;
by variable study;
table fellow*gold_standard / agree;
weight n / zeros;
ods output Kappa=k;
run;
proc transpose data=k out=ktable;
by variable study;
var nValue1;
id name1;
run;
proc sgplot data=ktable;
where variable ne "Ach_Obs";
BAND x=study lower=L_KAPPA upper=U_KAPPA / group=variable transparency=0.5;
series x=study y=_KAPPA_ / markers lineattrs=(pattern=solid thickness=2) group=variable;
xaxis type=discrete;
run;
```

PG

- Mark as New
- Bookmark
- Subscribe
- RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to ballardw

09-16-2016 02:54 PM

Accuracy was determined as the ratio of true predictions (sum of true positive and true negative) to all predictions (sum of false and false positives and negatives). (TP+TN)/ (TP+TN+FP+FN).

- Mark as New
- Bookmark
- Subscribe
- RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to smunigala

09-17-2016 05:19 AM

So it is repeated measure ? and it is longitude data ? Maybe you should post it at Stat forum,Steve could give you proc glimmix code. proc import datafile='/folders/myfolders/data_2_by_2.xlsx' out=have dbms=xlsx replace; run; data want; set have; if variable in ('Hiatal Hernia' 'Motor Pattern') and study in ('Baseline' '5' '10' '15' '20'); run; proc gee data = want; class study variable fellow gold_standard; model Count =variable study/ dist=poisson link=log ; lsmeans study/ ilink cl diff; repeated subject = variable*fellow*gold_standard / within = study type=unstr covb corrw; run;

- Mark as New
- Bookmark
- Subscribe
- RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to Ksharp

09-19-2016 11:27 AM

The PROC GEE code from @Ksharp should provide you everything you need to get started. I might have included some other factors in the model statement, but I am pressed for time this morning to really think clearly about it.

Steve Denham

- Mark as New
- Bookmark
- Subscribe
- RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to Ksharp

09-19-2016 12:50 PM

Steve, I received a notification that you have replied to the question I posed:

"I am interested in analyzing the generalized impulse response functions with Proc VARMAX. However, I am not sure how to program this with the Proc VAXMAX since none of the examples outlined in SAS indicated 'generalized'. I was wondering if someone would kindly help me with the program". However, I could not find your response to my question. Could you let me know where you placed your response. Thanks.

Mahmud

- Mark as New
- Bookmark
- Subscribe
- RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to mmansaray

09-20-2016 02:56 PM

Mahmud,

Since PROC VARMAX is in SAS/ETS, I moved the question to the Forecasting and Econometrics forum, where it should attract a more specialized audience. i didn't have any specific answer.

Steve Denham