topic Re: Looping a model over unique values of a variable and saving an output from each loop in SAS Programming

Looping a model over unique values of a variable and saving an output from each loop

Blain — Sun, 14 Apr 2019 01:42:46 GMT

I'm trying to run a survival model where I first need to create a vector (named 'time' here) which contains all unique t given c=1.

My final goal is to create a file or table where I can see the log likelihood for each value of t and find the t for which the log likelihood is maximum. My model is as follows:

proc phreg data = data; 
	model t*c(0) = Z1 Z2 / ties=Breslow; 
	dum = g-1; 
	if t > time then do Z2=dum; Z1=0; end; 
	else do Z2=0; Z1=dum; end; 
run;

Note that t=survival time, c=censoring status, g=group. I want to create a vector called 'time' which is the unique death times derived from t and c. Then I want to run the model for each value of 'time' and save the likelihood to create a table.

Re: Looping a model over unique values of a variable and saving an output from each loop

PaigeMiller — Sun, 14 Apr 2019 12:43:03 GMT

An outline of the solution:

proc sql noprint;
    select distinct t into :time separated by ' ' from data;
quit;

%macro dothis;
    %do i = 1 %to %sysfunc(countw(&time));
        %let thistime = %scan(&time,&i,%str( ));
        /* You will need to create an output data set from PROC PHREG */
        proc phreg data=data;
             ...
             if t > &thistime then do z2=dum; z1=0; end;
             ...
         run;
         data outputdataset;
               set outputdataset;
               time=&thistime;
         end;
         proc append base=all_outputs new=outputdataset;
         run;
    %end;
%mend;
%dothis

Re: Looping a model over unique values of a variable and saving an output from each loop

Blain — Mon, 15 Apr 2019 05:43:03 GMT

Thanks for your reply but I think I'm still struggling to append the data sets. I was able to save the value of the log likelihood only for the first loop. I attach the data file here and the codes I tried (without the attempt to append as it would give an error) are as follows:

data data_9_3uniq;
set data_9_3;
if c=1;
run;

proc sql noprint;
  select distinct t into :time from data_9_3uniq;
quit;

%macro dothis;
    %do i = 1 %to %sysfunc(countw(&time));
        %let thistime = %scan(&time,&i,%str( ));
		ods output FitStatistics=Fit; 
        proc phreg data=data_9_3;
             model t*c(0) = Z1 Z2 / ties=Breslow;
			 g_dum = g-1;
             if t > &thistime then do Z2=g_dum; Z1=0; end;
             else do Z2=0; Z1=g_dum; end;
         run;
         data outputdataset_&i;
               set Fit;
               time=&thistime;
			   keep time Criterion WithCovariates;
			   if Criterion="-2 LOG L";
		 run;
    %end;
%mend;
%dothis

I want to append the results from outputdataset_&i to a single file. So, the final data set will have unique times (&thistime) and the corresponding -2logLik or Lik.

Re: Looping a model over unique values of a variable and saving an output from each loop

PaigeMiller — Mon, 15 Apr 2019 10:50:19 GMT

@Blain wrote:

Thanks for your reply but I think I'm still struggling to append the data sets. I was able to save the value of the log likelihood only for the first loop. I attach the data file here and the codes I tried (without the attempt to append as it would give an error) are as follows:

My example code uses PROC APPEND.

Re: Looping a model over unique values of a variable and saving an output from each loop

Blain — Mon, 15 Apr 2019 18:27:50 GMT

I think I need to use proc append as you suggested. But when I run the codes it gives me an error and the results do not append successfully. I'm quite inexperienced so couldn't reproduce. It will be great if you can make time and use the attached data to show an example. I just added the codes that ran before the proc append step in my previous reply.

Re: Looping a model over unique values of a variable and saving an output from each loop

PaigeMiller — Mon, 15 Apr 2019 18:29:24 GMT

@Blain wrote:

But when I run the codes it gives me an error and the results do not append successfully.

Show us your code. Show us the LOG including the errors, by clicking on the {i} icon and pasting the log into that window.

Re: Looping a model over unique values of a variable and saving an output from each loop

Blain — Mon, 15 Apr 2019 19:33:26 GMT

Hi, I ran the following code without any error in the log this time:

data data_9_3uniq;
set data_9_3;
if c=1;
run;

proc sql noprint;
  select distinct t into :time from data_9_3uniq;
quit;

%macro dothis;
    %do i = 1 %to %sysfunc(countw(&time));
        %let thistime = %scan(&time,&i,%str( ));
		ods output FitStatistics=Fit; 
        proc phreg data=data_9_3;
             model t*c(0) = Z1 Z2 / ties=Breslow;
			 g_dum = g-1;
             if t > &thistime then do Z2=g_dum; Z1=0; end;
             else do Z2=0; Z1=g_dum; end;
         run;
         data outputdataset;
               set Fit;
               time=&thistime;
			   keep time Criterion WithCovariates;
			   if Criterion="-2 LOG L";
			   run;
         proc append base=all_outputs new=outputdataset;
         run;
    %end;
%mend;
%dothis

But the likelihoods didn't append into all_outputs. It only shows me 1 row for %time=1. Maybe I'm making some mistake.

Re: Looping a model over unique values of a variable and saving an output from each loop

PaigeMiller — Mon, 15 Apr 2019 19:40:58 GMT

As a diagnostic tool, you need to actually look at your data sets, with your own eyes, and see what is in them.

But, since I can't look at YOUR data sets, please see how many observations are in data_9_3uniq. Is there 1 observation? Is there 1 distinct value of t?

Re: Looping a model over unique values of a variable and saving an output from each loop

Blain — Mon, 15 Apr 2019 19:52:18 GMT

There are 80 unique t in my data set. So, time should have 80 unique values. I attached the data set for your reference.

Re: Looping a model over unique values of a variable and saving an output from each loop

PaigeMiller — Mon, 15 Apr 2019 19:58:09 GMT

Okay, I see the error, please go back to my original example code where it says:

proc sql noprint;
    select distinct t into :time separated by ' ' from data;
quit;

That's what you missed.

Re: Looping a model over unique values of a variable and saving an output from each loop

Blain — Mon, 15 Apr 2019 20:09:22 GMT

Thank you so much for your help!