Solved: Re: Simulate parameter estimates of the model for missing scenarios

Cruise · Posted 06-10-2019 01:49 PM

Dear SAS experts :

I'd like to visualize the effect of incomplete data on the parameter estimates of the model.
My gold-standard data has full date of diagnosis with complete month/day/yyyy fields.
The missing scenario that I'd like to assess is the 'month' and 'day' missing while 'year' is known.
Replacing unknown month/unknown day/known year by the 07/01/known year is a common practice in practice.
My hypothesis is: replacing unknown month/day by the mid point of the year July 1st (07/01/known year)
causes a significant bias in the survival estimates.

Therefore, I'd like to simulate the data of which 0-100% os the cases replaced by the 07/01/known year
compared to its gold-standard data I have with a complete diagnosis dates.

Desirable image is shown below to demonstrate the relationship between the 0 thru 100% data replaced by the
midpoint of the year and the corresponding bias in the parameter estimates of the survival model.

Bias is defined as the ratio of stage-specific parameter estimates between simulated data compared to
the parameter estimates of its gold-standard.

Proc Phreg in the codes produce outest=parest age-adjusted
stage-specific estimates. I imagine that if I run this model 100 times for 0 through 100 % of date of diagnosis
replaced by the July 1st for the unknown month/day fields then I would come up with the ratio of the missing/
gold standard parameter estimates and manually plot the imaginary plot shown in this post. Right?

Since there can be several missing scenarios that I'd like to test, running the hypothetical data for each
missing scenario hundred times each would be too time consuming.

I'll greatly appreciate your time and precious suggestions to automate this process?

Thanks zillions in advance. Please take a look at my proc phreg and mock data if needed.

/*SHOWING THE ORIGIN OF A MOCK DATASET*/
DATA HAVE; SET MYDATA;

date_of_diagnosis_mm1='7'; 
date_of_diagnosis_dd1='1'; 

DX=MDY(DATE_OF_DIAGNOSIS_MM,date_of_diagnosis_dd,DATE_OF_DIAGNOSIS_YYYY); /*GOLD STANDARD*/
DX_MID_YR=MDY(date_of_diagnosis_mm1,date_of_diagnosis_dd1,DATE_OF_DIAGNOSIS_YYYY); /*MID-YR REPLACEMENT*/

EXIT=MDY(date_of_death_mm,date_of_death_dd,date_of_death_yyyy);

duration_no_missing=sum(EXIT-DX);
duration_missing=sum(EXIT-DX_MID_YR);

FORMAT DX EXIT BEST12.; 
RUN;

/*RANDOM SAMPLE OF THE ACTUAL DATA*/
PROC SQL;
CREATE TABLE HAVE1 AS
SELECT * FROM HAVE
WHERE RANUNI(0)<0.007; /*N=143ROWS*/
QUIT;

DATA HAVE1;
INPUT ID status	date_of_death_mm	date_of_death_dd	date_of_death_yyyy	date_of_diagnosis_mm	date_of_diagnosis_dd	date_of_diagnosis_yyyy	agegrp	dx	exit	stage	DEATH	date_of_diagnosis_mm1	date_of_diagnosis_dd1	DX_MID_YR	duration_no_missing	duration_missing;
cards;
1	0	8	19	2005	3	22	2005	3	16517	16667	3	1	7	1	16618	150	49
2	0	6	19	2010	7	20	2005	2	16637	18432	4	1	7	1	16618	1795	1814
3	0	1	7	2006	8	29	2005	4	16677	16808	4	1	7	1	16618	131	190
4	0	5	24	2006	8	30	2005	3	16678	16945	4	1	7	1	16618	267	327
5	0	6	11	2007	11	7	2005	3	16747	17328	4	1	7	1	16618	581	710
6	0	12	27	2005	11	9	2005	4	16749	16797	4	1	7	1	16618	48	179
7	0	3	12	2006	11	25	2005	2	16765	16872	4	1	7	1	16618	107	254
8	0	11	3	2005	7	31	2005	3	16648	16743	4	1	7	1	16618	95	125
9	0	7	20	2005	3	9	2005	4	16504	16637	99	1	7	1	16618	133	19
10	0	12	21	2006	1	9	2006	3	16810	17156	4	1	7	1	16983	346	173
11	0	5	14	2006	2	25	2005	3	16492	16935	99	1	7	1	16618	443	317
12	0	9	8	2005	4	5	2005	4	16531	16687	99	1	7	1	16618	156	69
13	0	12	8	2005	9	12	2005	4	16691	16778	99	1	7	1	16618	87	160
14	0	4	14	2007	1	18	2006	2	16819	17270	4	1	7	1	16983	451	287
15	0	10	30	2009	2	8	2006	2	16840	18200	2	1	7	1	16983	1360	1217
16	0	3	2	2007	6	27	2006	1	16979	17227	3	1	7	1	16983	248	244
17	0	1	13	2007	10	23	2006	4	17097	17179	4	1	7	1	16983	82	196
18	0	12	18	2006	9	28	2006	3	17072	17153	4	1	7	1	16983	81	170
19	0	7	15	2007	12	15	2006	3	17150	17362	4	1	7	1	16983	212	379
20	0	7	11	2006	5	11	2006	4	16932	16993	4	1	7	1	16983	61	10
21	0	2	12	2008	7	24	2006	4	17006	17574	3	0	7	1	16983	568	591
22	0	1	18	2007	11	3	2006	4	17108	17184	4	0	7	1	16983	76	201
23	0	7	20	2010	6	5	2006	2	16957	18463	2	1	7	1	16983	1506	1480
24	0	11	12	2006	9	21	2006	1	17065	17117	4	1	7	1	16983	52	134
25	0	7	2	2007	1	22	2007	2	17188	17349	3	1	7	1	17348	161	1
26	0	3	16	2008	2	2	2007	3	17199	17607	4	1	7	1	17348	408	259
27	0	5	13	2009	3	12	2007	1	17237	18030	4	1	7	1	17348	793	682
28	0	8	28	2007	3	16	2007	4	17241	17406	3	1	7	1	17348	165	58
29	0	2	16	2007	9	26	2006	4	17070	17213	4	1	7	1	16983	143	230
30	0	10	13	2008	5	14	2007	4	17300	17818	3	1	7	1	17348	518	470
31	0	12	22	2007	8	13	2007	3	17391	17522	4	1	7	1	17348	131	174
32	0	4	13	2008	9	7	2007	3	17416	17635	4	1	7	1	17348	219	287
33	0	12	30	2007	7	9	2007	3	17356	17530	3	0	7	1	17348	174	182
34	0	12	1	2006	7	14	2006	4	16996	17136	99	1	7	1	16983	140	153
35	0	2	7	2010	9	15	2007	3	17424	18300	3	1	7	1	17348	876	952
36	0	4	29	2008	6	29	2007	3	17346	17651	3	1	7	1	17348	305	303
37	0	12	4	2007	12	1	2007	3	17501	17504	4	1	7	1	17348	3	156
38	0	9	18	2007	7	16	2007	3	17363	17427	4	1	7	1	17348	64	79
39	0	12	2	2007	10	18	2007	3	17457	17502	3	1	7	1	17348	45	154
40	0	6	23	2008	12	3	2007	4	17503	17706	3	1	7	1	17348	203	358
41	0	8	23	2008	2	4	2008	2	17566	17767	4	1	7	1	17714	201	53
42	0	6	6	2009	4	14	2008	3	17636	18054	4	1	7	1	17714	418	340
43	0	8	21	2011	5	16	2008	3	17668	18860	3	1	7	1	17714	1192	1146
44	0	1	14	2009	9	24	2008	4	17799	17911	4	1	7	1	17714	112	197
45	0	7	9	2009	4	22	2009	4	18009	18087	3	1	7	1	18079	78	8
46	0	9	28	2010	3	17	2009	2	17973	18533	2	0	7	1	18079	560	454
47	0	10	6	2009	4	30	2009	3	18017	18176	4	0	7	1	18079	159	97
48	0	7	10	2009	5	29	2009	4	18046	18088	3	0	7	1	18079	42	9
49	0	8	9	2010	6	4	2009	3	18052	18483	3	1	7	1	18079	431	404
50	0	11	25	2009	7	10	2009	3	18088	18226	4	1	7	1	18079	138	147
51	0	9	16	2009	5	22	2009	4	18039	18156	3	0	7	1	18079	117	77
52	0	4	28	2010	8	21	2009	3	18130	18380	4	1	7	1	18079	250	301
53	0	1	30	2010	9	15	2009	3	18155	18292	2	1	7	1	18079	137	213
54	0	1	21	2010	9	2	2009	3	18142	18283	3	1	7	1	18079	141	204
55	0	10	18	2010	10	9	2009	3	18179	18553	3	0	7	1	18079	374	474
56	0	2	7	2010	10	24	2009	3	18194	18300	4	1	7	1	18079	106	221
57	0	1	26	2010	10	6	2009	3	18176	18288	4	1	7	1	18079	112	209
58	0	2	24	2010	11	19	2009	2	18220	18317	4	1	7	1	18079	97	238
59	0	12	28	2012	11	17	2009	3	18218	19355	4	1	7	1	18079	1137	1276
60	0	6	14	2010	12	2	2009	3	18233	18427	4	1	7	1	18079	194	348
61	0	3	16	2012	11	16	2009	4	18217	19068	3	1	7	1	18079	851	989
62	0	3	20	2010	7	30	2009	3	18108	18341	3	1	7	1	18079	233	262
63	0	5	10	2011	2	23	2010	2	18316	18757	4	1	7	1	18444	441	313
64	0	6	8	2012	3	12	2010	2	18333	19152	4	1	7	1	18444	819	708
65	0	7	10	2010	5	6	2010	4	18388	18453	3	1	7	1	18444	65	9
66	0	7	11	2010	5	25	2010	3	18407	18454	3	1	7	1	18444	47	10
67	0	3	7	2011	7	30	2010	3	18473	18693	4	1	7	1	18444	220	249
68	0	2	7	2012	7	18	2010	2	18461	19030	4	1	7	1	18444	569	586
69	0	12	18	2010	9	3	2010	3	18508	18614	2	0	7	1	18444	106	170
70	0	11	24	2013	11	9	2010	4	18575	19686	3	1	7	1	18444	1111	1242
71	0	4	30	2012	11	12	2010	3	18578	19113	3	1	7	1	18444	535	669
72	0	2	21	2012	11	12	2010	3	18578	19044	3	1	7	1	18444	466	600
73	0	6	12	2011	12	14	2010	3	18610	18790	4	1	7	1	18444	180	346
74	0	7	17	2011	12	20	2010	4	18616	18825	3	1	7	1	18444	209	381
75	0	7	4	2013	11	3	2010	3	18569	19543	3	1	7	1	18444	974	1099
76	0	12	22	2012	10	22	2010	3	18557	19349	3	1	7	1	18444	792	905
77	0	11	28	2013	1	25	2011	3	18652	19690	4	1	7	1	18809	1038	881
78	0	12	27	2011	6	3	2011	3	18781	18988	3	0	7	1	18809	207	179
79	0	12	5	2011	5	2	2011	4	18749	18966	3	1	7	1	18809	217	157
80	0	7	21	2011	5	23	2011	3	18770	18829	4	1	7	1	18809	59	20
81	0	2	27	2014	7	13	2011	3	18821	19781	3	0	7	1	18809	960	972
82	0	12	22	2011	10	12	2011	4	18912	18983	4	1	7	1	18809	71	174
83	0	2	1	2012	9	10	2011	4	18880	19024	4	1	7	1	18809	144	215
84	0	7	14	2012	9	6	2011	4	18876	19188	3	1	7	1	18809	312	379
85	0	12	2	2012	12	9	2011	4	18970	19329	3	1	7	1	18809	359	520
86	0	10	29	2012	12	5	2011	2	18966	19295	4	1	7	1	18809	329	486
87	0	7	10	2015	11	17	2011	1	18948	20279	4	1	7	1	18809	1331	1470
88	0	3	19	2012	12	28	2011	2	18989	19071	4	1	7	1	18809	82	262
89	0	11	16	2012	1	26	2012	3	19018	19313	2	0	7	1	19175	295	138
90	0	2	8	2014	1	5	2012	4	18997	19762	3	1	7	1	19175	765	587
91	0	8	27	2013	2	21	2012	3	19044	19597	4	1	7	1	19175	553	422
92	0	5	2	2013	2	20	2012	3	19043	19480	3	1	7	1	19175	437	305
93	0	2	12	2013	4	13	2012	4	19096	19401	3	1	7	1	19175	305	226
94	0	7	16	2013	4	16	2012	3	19099	19555	2	1	7	1	19175	456	380
95	0	3	4	2013	5	10	2012	3	19123	19421	4	1	7	1	19175	298	246
96	0	7	23	2012	5	14	2012	3	19127	19197	3	1	7	1	19175	70	22
97	0	7	6	2012	5	31	2012	3	19144	19180	4	1	7	1	19175	36	5
98	0	12	22	2012	6	9	2012	2	19153	19349	3	1	7	1	19175	196	174
99	0	6	19	2013	6	28	2012	3	19172	19528	3	1	7	1	19175	356	353
100	0	6	5	2013	8	14	2012	4	19219	19514	3	1	7	1	19175	295	339
101	0	11	5	2012	10	24	2012	4	19290	19302	3	1	7	1	19175	12	127
102	0	8	1	2013	11	9	2012	2	19306	19571	4	1	7	1	19175	265	396
103	0	8	14	2013	11	30	2012	4	19327	19584	4	1	7	1	19175	257	409
104	0	8	3	2012	4	17	2012	3	19100	19208	4	1	7	1	19175	108	33
105	0	3	19	2013	11	5	2012	4	19302	19436	4	1	7	1	19175	134	261
106	0	7	31	2014	11	26	2012	4	19323	19935	4	1	7	1	19175	612	760
107	0	8	18	2014	12	21	2012	3	19348	19953	4	1	7	1	19175	605	778
108	0	11	22	2013	1	10	2013	3	19368	19684	4	1	7	1	19540	316	144
109	0	11	3	2014	1	17	2013	4	19375	20030	3	1	7	1	19540	655	490
110	0	9	14	2013	1	23	2013	4	19381	19615	3	1	7	1	19540	234	75
111	0	12	6	2013	1	30	2013	2	19388	19698	3	1	7	1	19540	310	158
112	0	2	17	2014	3	8	2013	1	19425	19771	4	1	7	1	19540	346	231
113	0	3	4	2014	7	18	2013	2	19557	19786	4	1	7	1	19540	229	246
114	0	8	2	2015	6	11	2013	3	19520	20302	3	1	7	1	19540	782	762
115	0	3	22	2014	10	27	2009	3	18197	19804	2	1	7	1	18079	1607	1725
116	0	11	15	2013	9	26	2013	4	19627	19677	99	1	7	1	19540	50	137
117	0	5	29	2014	11	13	2013	2	19675	19872	4	1	7	1	19540	197	332
118	0	11	6	2014	9	5	2013	3	19606	20033	3	1	7	1	19540	427	493
119	0	7	11	2012	5	10	2012	4	19123	19185	4	1	7	1	19175	62	10
120	0	9	28	2016	12	4	2012	4	19331	20725	3	1	7	1	19175	1394	1550
121	0	8	31	2014	12	21	2013	3	19713	19966	4	1	7	1	19540	253	426
122	0	5	5	2016	1	17	2014	2	19740	20579	4	1	7	1	19905	839	674
123	0	8	1	2015	3	3	2014	2	19785	20301	3	1	7	1	19905	516	396
124	0	11	29	2014	5	16	2014	4	19859	20056	3	1	7	1	19905	197	151
125	0	6	11	2015	7	2	2014	2	19906	20250	3	1	7	1	19905	344	345
126	0	7	18	2015	9	24	2014	3	19990	20287	4	1	7	1	19905	297	382
127	0	7	24	2016	10	30	2014	3	20026	20659	4	1	7	1	19905	633	754
128	0	7	5	2015	12	9	2014	3	20066	20274	4	1	7	1	19905	208	369
129	0	11	19	2015	2	3	2015	1	20122	20411	3	1	7	1	20270	289	141
130	0	10	31	2015	1	8	2015	2	20096	20392	4	1	7	1	20270	296	122
131	0	3	15	2016	3	26	2015	2	20173	20528	4	1	7	1	20270	355	258
132	0	7	4	2015	3	10	2015	3	20157	20273	3	1	7	1	20270	116	3
133	0	8	9	2015	3	17	2015	2	20164	20309	4	1	7	1	20270	145	39
134	0	12	8	2016	5	22	2015	3	20230	20796	2	1	7	1	20270	566	526
135	0	8	15	2016	5	27	2015	4	20235	20681	4	1	7	1	20270	446	411
136	0	7	11	2015	5	8	2015	3	20216	20280	4	1	7	1	20270	64	10
137	0	10	19	2016	7	1	2015	4	20270	20746	3	0	7	1	20270	476	476
138	0	10	28	2015	9	1	2015	3	20332	20389	4	1	7	1	20270	57	119
139	0	2	14	2016	8	6	2015	3	20306	20498	4	1	7	1	20270	192	228
140	0	12	21	2016	11	17	2015	3	20409	20809	4	1	7	1	20270	400	539
141	0	9	12	2016	10	16	2015	3	20377	20709	3	1	7	1	20270	332	439
142	0	4	26	2016	3	2	2015	4	20149	20570	99	0	7	1	20270	421	300
143	0	7	28	2016	8	25	2015	2	20325	20663	3	1	7	1	20270	338	393
;
RUN;

/*SURVIVAL ANALYSIS ON THE ACTUAL DATA WITH NO MISSING*/
PROC PHREG DATA = HAVE1 OUTEST=PAREST; 
CLASS STAGE(REF='1') AGEGRP(REF='1')/PARAM=REF; 
MODEL duration_no_missing*DEATH(0) = AGEGRP STAGE/RL EVENTCODE=1; 
BASELINE OUT = ESTIMATE; 
output out=resOut; 
RUN;
/*simulate duration_missing for the 0-100% of the data*/
PROC PHREG DATA = HAVE1 OUTEST=PAREST; 
CLASS STAGE(REF='1') AGEGRP(REF='1')/PARAM=REF; 
MODEL duration_missing*DEATH(0) = AGEGRP STAGE/RL EVENTCODE=1; 
BASELINE OUT = ESTIMATE; 
output out=resOut; 
RUN;

Reeza · Posted 06-10-2019 10:33 PM

PLEASE RUN YOUR CODE BEFORE POSTING.

You don't have any Stage = 1 in your data sets but you set those as the reference group, you'll need to fix that. I spent ten minutes trying to ensure my sample had all values and figuring out why it wouldn't work before I checked the original data set.

I DID NOT take a close look at your PHREG code or HAVE1 logic to ensure it was correct, that's up to you.

I highly recommend you do some replicates as well, since a single sample can vary quite a bit. In this case I was running 100 replicates.

I was calling the macro manually at the end because I'm just testing. For running this fully, once you've got it finalized, I recommend using a data _null_ step and call execute rather than manual macro calls.

And I haven't created any graphs or summaries, this will give you the information to do that though.



DATA HAVE1;
INPUT ID status date_of_death_mm    date_of_death_dd    date_of_death_yyyy  date_of_diagnosis_mm    date_of_diagnosis_dd    date_of_diagnosis_yyyy  agegrp  dx  exit    stage   DEATH   date_of_diagnosis_mm1   date_of_diagnosis_dd1   DX_MID_YR   duration_no_missing duration_missing;
cards;
1   0   8   19  2005    3   22  2005    3   16517   16667   3   1   7   1   16618   150 49
2   0   6   19  2010    7   20  2005    2   16637   18432   4   1   7   1   16618   1795    1814
3   0   1   7   2006    8   29  2005    4   16677   16808   4   1   7   1   16618   131 190
4   0   5   24  2006    8   30  2005    3   16678   16945   4   1   7   1   16618   267 327
5   0   6   11  2007    11  7   2005    3   16747   17328   4   1   7   1   16618   581 710
6   0   12  27  2005    11  9   2005    4   16749   16797   4   1   7   1   16618   48  179
7   0   3   12  2006    11  25  2005    2   16765   16872   4   1   7   1   16618   107 254
8   0   11  3   2005    7   31  2005    3   16648   16743   4   1   7   1   16618   95  125
9   0   7   20  2005    3   9   2005    4   16504   16637   99  1   7   1   16618   133 19
10  0   12  21  2006    1   9   2006    3   16810   17156   4   1   7   1   16983   346 173
11  0   5   14  2006    2   25  2005    3   16492   16935   99  1   7   1   16618   443 317
12  0   9   8   2005    4   5   2005    4   16531   16687   99  1   7   1   16618   156 69
13  0   12  8   2005    9   12  2005    4   16691   16778   99  1   7   1   16618   87  160
14  0   4   14  2007    1   18  2006    2   16819   17270   4   1   7   1   16983   451 287
15  0   10  30  2009    2   8   2006    2   16840   18200   2   1   7   1   16983   1360    1217
16  0   3   2   2007    6   27  2006    1   16979   17227   3   1   7   1   16983   248 244
17  0   1   13  2007    10  23  2006    4   17097   17179   4   1   7   1   16983   82  196
18  0   12  18  2006    9   28  2006    3   17072   17153   4   1   7   1   16983   81  170
19  0   7   15  2007    12  15  2006    3   17150   17362   4   1   7   1   16983   212 379
20  0   7   11  2006    5   11  2006    4   16932   16993   4   1   7   1   16983   61  10
21  0   2   12  2008    7   24  2006    4   17006   17574   3   0   7   1   16983   568 591
22  0   1   18  2007    11  3   2006    4   17108   17184   4   0   7   1   16983   76  201
23  0   7   20  2010    6   5   2006    2   16957   18463   2   1   7   1   16983   1506    1480
24  0   11  12  2006    9   21  2006    1   17065   17117   4   1   7   1   16983   52  134
25  0   7   2   2007    1   22  2007    2   17188   17349   3   1   7   1   17348   161 1
26  0   3   16  2008    2   2   2007    3   17199   17607   4   1   7   1   17348   408 259
27  0   5   13  2009    3   12  2007    1   17237   18030   4   1   7   1   17348   793 682
28  0   8   28  2007    3   16  2007    4   17241   17406   3   1   7   1   17348   165 58
29  0   2   16  2007    9   26  2006    4   17070   17213   4   1   7   1   16983   143 230
30  0   10  13  2008    5   14  2007    4   17300   17818   3   1   7   1   17348   518 470
31  0   12  22  2007    8   13  2007    3   17391   17522   4   1   7   1   17348   131 174
32  0   4   13  2008    9   7   2007    3   17416   17635   4   1   7   1   17348   219 287
33  0   12  30  2007    7   9   2007    3   17356   17530   3   0   7   1   17348   174 182
34  0   12  1   2006    7   14  2006    4   16996   17136   99  1   7   1   16983   140 153
35  0   2   7   2010    9   15  2007    3   17424   18300   3   1   7   1   17348   876 952
36  0   4   29  2008    6   29  2007    3   17346   17651   3   1   7   1   17348   305 303
37  0   12  4   2007    12  1   2007    3   17501   17504   4   1   7   1   17348   3   156
38  0   9   18  2007    7   16  2007    3   17363   17427   4   1   7   1   17348   64  79
39  0   12  2   2007    10  18  2007    3   17457   17502   3   1   7   1   17348   45  154
40  0   6   23  2008    12  3   2007    4   17503   17706   3   1   7   1   17348   203 358
41  0   8   23  2008    2   4   2008    2   17566   17767   4   1   7   1   17714   201 53
42  0   6   6   2009    4   14  2008    3   17636   18054   4   1   7   1   17714   418 340
43  0   8   21  2011    5   16  2008    3   17668   18860   3   1   7   1   17714   1192    1146
44  0   1   14  2009    9   24  2008    4   17799   17911   4   1   7   1   17714   112 197
45  0   7   9   2009    4   22  2009    4   18009   18087   3   1   7   1   18079   78  8
46  0   9   28  2010    3   17  2009    2   17973   18533   2   0   7   1   18079   560 454
47  0   10  6   2009    4   30  2009    3   18017   18176   4   0   7   1   18079   159 97
48  0   7   10  2009    5   29  2009    4   18046   18088   3   0   7   1   18079   42  9
49  0   8   9   2010    6   4   2009    3   18052   18483   3   1   7   1   18079   431 404
50  0   11  25  2009    7   10  2009    3   18088   18226   4   1   7   1   18079   138 147
51  0   9   16  2009    5   22  2009    4   18039   18156   3   0   7   1   18079   117 77
52  0   4   28  2010    8   21  2009    3   18130   18380   4   1   7   1   18079   250 301
53  0   1   30  2010    9   15  2009    3   18155   18292   2   1   7   1   18079   137 213
54  0   1   21  2010    9   2   2009    3   18142   18283   3   1   7   1   18079   141 204
55  0   10  18  2010    10  9   2009    3   18179   18553   3   0   7   1   18079   374 474
56  0   2   7   2010    10  24  2009    3   18194   18300   4   1   7   1   18079   106 221
57  0   1   26  2010    10  6   2009    3   18176   18288   4   1   7   1   18079   112 209
58  0   2   24  2010    11  19  2009    2   18220   18317   4   1   7   1   18079   97  238
59  0   12  28  2012    11  17  2009    3   18218   19355   4   1   7   1   18079   1137    1276
60  0   6   14  2010    12  2   2009    3   18233   18427   4   1   7   1   18079   194 348
61  0   3   16  2012    11  16  2009    4   18217   19068   3   1   7   1   18079   851 989
62  0   3   20  2010    7   30  2009    3   18108   18341   3   1   7   1   18079   233 262
63  0   5   10  2011    2   23  2010    2   18316   18757   4   1   7   1   18444   441 313
64  0   6   8   2012    3   12  2010    2   18333   19152   4   1   7   1   18444   819 708
65  0   7   10  2010    5   6   2010    4   18388   18453   3   1   7   1   18444   65  9
66  0   7   11  2010    5   25  2010    3   18407   18454   3   1   7   1   18444   47  10
67  0   3   7   2011    7   30  2010    3   18473   18693   4   1   7   1   18444   220 249
68  0   2   7   2012    7   18  2010    2   18461   19030   4   1   7   1   18444   569 586
69  0   12  18  2010    9   3   2010    3   18508   18614   2   0   7   1   18444   106 170
70  0   11  24  2013    11  9   2010    4   18575   19686   3   1   7   1   18444   1111    1242
71  0   4   30  2012    11  12  2010    3   18578   19113   3   1   7   1   18444   535 669
72  0   2   21  2012    11  12  2010    3   18578   19044   3   1   7   1   18444   466 600
73  0   6   12  2011    12  14  2010    3   18610   18790   4   1   7   1   18444   180 346
74  0   7   17  2011    12  20  2010    4   18616   18825   3   1   7   1   18444   209 381
75  0   7   4   2013    11  3   2010    3   18569   19543   3   1   7   1   18444   974 1099
76  0   12  22  2012    10  22  2010    3   18557   19349   3   1   7   1   18444   792 905
77  0   11  28  2013    1   25  2011    3   18652   19690   4   1   7   1   18809   1038    881
78  0   12  27  2011    6   3   2011    3   18781   18988   3   0   7   1   18809   207 179
79  0   12  5   2011    5   2   2011    4   18749   18966   3   1   7   1   18809   217 157
80  0   7   21  2011    5   23  2011    3   18770   18829   4   1   7   1   18809   59  20
81  0   2   27  2014    7   13  2011    3   18821   19781   3   0   7   1   18809   960 972
82  0   12  22  2011    10  12  2011    4   18912   18983   4   1   7   1   18809   71  174
83  0   2   1   2012    9   10  2011    4   18880   19024   4   1   7   1   18809   144 215
84  0   7   14  2012    9   6   2011    4   18876   19188   3   1   7   1   18809   312 379
85  0   12  2   2012    12  9   2011    4   18970   19329   3   1   7   1   18809   359 520
86  0   10  29  2012    12  5   2011    2   18966   19295   4   1   7   1   18809   329 486
87  0   7   10  2015    11  17  2011    1   18948   20279   4   1   7   1   18809   1331    1470
88  0   3   19  2012    12  28  2011    2   18989   19071   4   1   7   1   18809   82  262
89  0   11  16  2012    1   26  2012    3   19018   19313   2   0   7   1   19175   295 138
90  0   2   8   2014    1   5   2012    4   18997   19762   3   1   7   1   19175   765 587
91  0   8   27  2013    2   21  2012    3   19044   19597   4   1   7   1   19175   553 422
92  0   5   2   2013    2   20  2012    3   19043   19480   3   1   7   1   19175   437 305
93  0   2   12  2013    4   13  2012    4   19096   19401   3   1   7   1   19175   305 226
94  0   7   16  2013    4   16  2012    3   19099   19555   2   1   7   1   19175   456 380
95  0   3   4   2013    5   10  2012    3   19123   19421   4   1   7   1   19175   298 246
96  0   7   23  2012    5   14  2012    3   19127   19197   3   1   7   1   19175   70  22
97  0   7   6   2012    5   31  2012    3   19144   19180   4   1   7   1   19175   36  5
98  0   12  22  2012    6   9   2012    2   19153   19349   3   1   7   1   19175   196 174
99  0   6   19  2013    6   28  2012    3   19172   19528   3   1   7   1   19175   356 353
100 0   6   5   2013    8   14  2012    4   19219   19514   3   1   7   1   19175   295 339
101 0   11  5   2012    10  24  2012    4   19290   19302   3   1   7   1   19175   12  127
102 0   8   1   2013    11  9   2012    2   19306   19571   4   1   7   1   19175   265 396
103 0   8   14  2013    11  30  2012    4   19327   19584   4   1   7   1   19175   257 409
104 0   8   3   2012    4   17  2012    3   19100   19208   4   1   7   1   19175   108 33
105 0   3   19  2013    11  5   2012    4   19302   19436   4   1   7   1   19175   134 261
106 0   7   31  2014    11  26  2012    4   19323   19935   4   1   7   1   19175   612 760
107 0   8   18  2014    12  21  2012    3   19348   19953   4   1   7   1   19175   605 778
108 0   11  22  2013    1   10  2013    3   19368   19684   4   1   7   1   19540   316 144
109 0   11  3   2014    1   17  2013    4   19375   20030   3   1   7   1   19540   655 490
110 0   9   14  2013    1   23  2013    4   19381   19615   3   1   7   1   19540   234 75
111 0   12  6   2013    1   30  2013    2   19388   19698   3   1   7   1   19540   310 158
112 0   2   17  2014    3   8   2013    1   19425   19771   4   1   7   1   19540   346 231
113 0   3   4   2014    7   18  2013    2   19557   19786   4   1   7   1   19540   229 246
114 0   8   2   2015    6   11  2013    3   19520   20302   3   1   7   1   19540   782 762
115 0   3   22  2014    10  27  2009    3   18197   19804   2   1   7   1   18079   1607    1725
116 0   11  15  2013    9   26  2013    4   19627   19677   99  1   7   1   19540   50  137
117 0   5   29  2014    11  13  2013    2   19675   19872   4   1   7   1   19540   197 332
118 0   11  6   2014    9   5   2013    3   19606   20033   3   1   7   1   19540   427 493
119 0   7   11  2012    5   10  2012    4   19123   19185   4   1   7   1   19175   62  10
120 0   9   28  2016    12  4   2012    4   19331   20725   3   1   7   1   19175   1394    1550
121 0   8   31  2014    12  21  2013    3   19713   19966   4   1   7   1   19540   253 426
122 0   5   5   2016    1   17  2014    2   19740   20579   4   1   7   1   19905   839 674
123 0   8   1   2015    3   3   2014    2   19785   20301   3   1   7   1   19905   516 396
124 0   11  29  2014    5   16  2014    4   19859   20056   3   1   7   1   19905   197 151
125 0   6   11  2015    7   2   2014    2   19906   20250   3   1   7   1   19905   344 345
126 0   7   18  2015    9   24  2014    3   19990   20287   4   1   7   1   19905   297 382
127 0   7   24  2016    10  30  2014    3   20026   20659   4   1   7   1   19905   633 754
128 0   7   5   2015    12  9   2014    3   20066   20274   4   1   7   1   19905   208 369
129 0   11  19  2015    2   3   2015    1   20122   20411   3   1   7   1   20270   289 141
130 0   10  31  2015    1   8   2015    2   20096   20392   4   1   7   1   20270   296 122
131 0   3   15  2016    3   26  2015    2   20173   20528   4   1   7   1   20270   355 258
132 0   7   4   2015    3   10  2015    3   20157   20273   3   1   7   1   20270   116 3
133 0   8   9   2015    3   17  2015    2   20164   20309   4   1   7   1   20270   145 39
134 0   12  8   2016    5   22  2015    3   20230   20796   2   1   7   1   20270   566 526
135 0   8   15  2016    5   27  2015    4   20235   20681   4   1   7   1   20270   446 411
136 0   7   11  2015    5   8   2015    3   20216   20280   4   1   7   1   20270   64  10
137 0   10  19  2016    7   1   2015    4   20270   20746   3   0   7   1   20270   476 476
138 0   10  28  2015    9   1   2015    3   20332   20389   4   1   7   1   20270   57  119
139 0   2   14  2016    8   6   2015    3   20306   20498   4   1   7   1   20270   192 228
140 0   12  21  2016    11  17  2015    3   20409   20809   4   1   7   1   20270   400 539
141 0   9   12  2016    10  16  2015    3   20377   20709   3   1   7   1   20270   332 439
142 0   4   26  2016    3   2   2015    4   20149   20570   99  0   7   1   20270   421 300
143 0   7   28  2016    8   25  2015    2   20325   20663   3   1   7   1   20270   338 393
;
RUN;

*sort for demo;
proc sort data=have1; 
by stage;
run;


%macro surveyLoop(sample=);

*calculate sample rate;
%let sampRate = %sysevalf(&sample./100);

*create sample;
ods select none;
proc surveyselect data=have1 
                   samprate=&sampRate. 
                   reps=100 
                   out=_sample 
                   seed=995
                   outall;
strata stage;*ensures stage is present for all values - not needed in prod;
run;

*set duration to values if selected;
data _sample;
set _sample;

*assign values as needed;
if selected = 1 then duration = duration_missing;
else duration=duration_no_missing;
run;

*sort for modeling procedures next;
proc sort data=_sample;
by replicate;
run;


/*SURVIVAL ANALYSIS ON THE ACTUAL DATA WITH NO MISSING*/
ods output parameterEstimates = PE_MISS;
PROC PHREG DATA = _sample; 
by replicate;
CLASS STAGE(REF='99') AGEGRP(REF='1')/PARAM=REF; 
MODEL duration_no_missing*DEATH(0) = AGEGRP STAGE/RL EVENTCODE=1; 
*BASELINE OUT = ESTIMATE SURVIVAL = SURVIVAL; 
*output out=resOut; 
RUN;


ods output parameterEstimates = PE_ALL;
/*simulate duration_missing for the 0-100% of the data*/
PROC PHREG DATA = _sample ; 
by replicate;
CLASS STAGE(REF='99') AGEGRP(REF='1')/PARAM=REF; 
MODEL duration*DEATH(0) = AGEGRP STAGE/RL EVENTCODE=1; 
RUN;


*combines results and labels estimates;
data PE_COMBO;
set pe_miss (in=t1) pe_all (in=t2);
length estType $15.;
PCT_MISS = &sample/100;
if t1 then estType = 'Missing';
else estType = 'Full Sample';
run;

*omit if no replicates;
*if replicates calculates average of HR + STDERR for bootstrap approach;
proc means data=pe_combo NWAY N MEAN STD STDERR;
class estType parameter classval0 pct_miss;
var estimate hazardratio;
ods output summary = pe_summary;
run;
ods select all;

*append to main data sets;
proc append base=results_combined data=pe_combo force;
run;

proc append base=results_summary data=pe_summary force;
run;

*clean up after;
proc datasets lib=work nodetails nolist;
delete pe_: _sample;
run;quit;

%let sampRate =;

%mend;


*call macro for different sizes;
%surveyLoop(sample=50);
%surveyLoop(sample=60);
%surveyLoop(sample=70);

View solution in original post

Reeza · Posted 06-10-2019 02:54 PM

Replacing unknown month/unknown day/known year by the 07/01/known year is a common practice in practice.

Just in case your literature review missed this, demography/social sciences has done a lot of study around this date/practice. July 1 is often used because it's the midpoint and allows averaging out for people who are below and above that date. It's also consistently the date used for census and population estimates so allows the dates to align. I'm sure your lit review has uncovered this. There's also some corrections used to account for it, but I can't recall the specifics.

Good luck with your simulation.

Cruise · Posted 06-10-2019 03:00 PM

Thanks Reeza. I agree. Common practice indeed.

Cruise · Posted 06-10-2019 04:45 PM

@PaigeMiller please look at this simulation problem when you have a time. The mark0-mark100 that you just provided a solution was relevant to this post. Eventually this is the simulation I want. However, mark0-mark100 just created is to come up with a preliminary data at 0, 5, 25, 50, 75 and 100% of the missing in the dataset.

PGStats · Posted 06-10-2019 04:57 PM

The graph would be easier to interpret if you switched the X and Y axes.

Your code doesn't show how duration and agegrp are adjusted for partial dates.

You could start with replacement rates = 10, ... , 90%, with 20 random replicates. That should tell you if your hypothesis is plausible.

PG

Cruise · Posted 06-10-2019 05:34 PM

@PGStats thank you. the data is the gold-standard data where all cases have full dates with no missing. therefore, agegrp is not adjusted. Duration for the replaced data will be calculated from the replaced date of diagnosis., such as: date of death-July/01/known year. I'd like to simulate 0-100% of the data for incomplete dates thus need to be replaced by July 1st of the given year. I'm working on the few select % of the data for missing to test the hypothesis as you suggested.

Cruise · Posted 06-10-2019 06:21 PM

@PGStats I flipped the axis in the image and updated the mock data. Thanks for the pointers. I appreciate your help if you happen to have time to dedicate for this problem. I like questions too which I learn a lot.

Reeza · Posted 06-10-2019 10:33 PM

PLEASE RUN YOUR CODE BEFORE POSTING.

You don't have any Stage = 1 in your data sets but you set those as the reference group, you'll need to fix that. I spent ten minutes trying to ensure my sample had all values and figuring out why it wouldn't work before I checked the original data set.

I DID NOT take a close look at your PHREG code or HAVE1 logic to ensure it was correct, that's up to you.

I highly recommend you do some replicates as well, since a single sample can vary quite a bit. In this case I was running 100 replicates.

I was calling the macro manually at the end because I'm just testing. For running this fully, once you've got it finalized, I recommend using a data _null_ step and call execute rather than manual macro calls.

And I haven't created any graphs or summaries, this will give you the information to do that though.



DATA HAVE1;
INPUT ID status date_of_death_mm    date_of_death_dd    date_of_death_yyyy  date_of_diagnosis_mm    date_of_diagnosis_dd    date_of_diagnosis_yyyy  agegrp  dx  exit    stage   DEATH   date_of_diagnosis_mm1   date_of_diagnosis_dd1   DX_MID_YR   duration_no_missing duration_missing;
cards;
1   0   8   19  2005    3   22  2005    3   16517   16667   3   1   7   1   16618   150 49
2   0   6   19  2010    7   20  2005    2   16637   18432   4   1   7   1   16618   1795    1814
3   0   1   7   2006    8   29  2005    4   16677   16808   4   1   7   1   16618   131 190
4   0   5   24  2006    8   30  2005    3   16678   16945   4   1   7   1   16618   267 327
5   0   6   11  2007    11  7   2005    3   16747   17328   4   1   7   1   16618   581 710
6   0   12  27  2005    11  9   2005    4   16749   16797   4   1   7   1   16618   48  179
7   0   3   12  2006    11  25  2005    2   16765   16872   4   1   7   1   16618   107 254
8   0   11  3   2005    7   31  2005    3   16648   16743   4   1   7   1   16618   95  125
9   0   7   20  2005    3   9   2005    4   16504   16637   99  1   7   1   16618   133 19
10  0   12  21  2006    1   9   2006    3   16810   17156   4   1   7   1   16983   346 173
11  0   5   14  2006    2   25  2005    3   16492   16935   99  1   7   1   16618   443 317
12  0   9   8   2005    4   5   2005    4   16531   16687   99  1   7   1   16618   156 69
13  0   12  8   2005    9   12  2005    4   16691   16778   99  1   7   1   16618   87  160
14  0   4   14  2007    1   18  2006    2   16819   17270   4   1   7   1   16983   451 287
15  0   10  30  2009    2   8   2006    2   16840   18200   2   1   7   1   16983   1360    1217
16  0   3   2   2007    6   27  2006    1   16979   17227   3   1   7   1   16983   248 244
17  0   1   13  2007    10  23  2006    4   17097   17179   4   1   7   1   16983   82  196
18  0   12  18  2006    9   28  2006    3   17072   17153   4   1   7   1   16983   81  170
19  0   7   15  2007    12  15  2006    3   17150   17362   4   1   7   1   16983   212 379
20  0   7   11  2006    5   11  2006    4   16932   16993   4   1   7   1   16983   61  10
21  0   2   12  2008    7   24  2006    4   17006   17574   3   0   7   1   16983   568 591
22  0   1   18  2007    11  3   2006    4   17108   17184   4   0   7   1   16983   76  201
23  0   7   20  2010    6   5   2006    2   16957   18463   2   1   7   1   16983   1506    1480
24  0   11  12  2006    9   21  2006    1   17065   17117   4   1   7   1   16983   52  134
25  0   7   2   2007    1   22  2007    2   17188   17349   3   1   7   1   17348   161 1
26  0   3   16  2008    2   2   2007    3   17199   17607   4   1   7   1   17348   408 259
27  0   5   13  2009    3   12  2007    1   17237   18030   4   1   7   1   17348   793 682
28  0   8   28  2007    3   16  2007    4   17241   17406   3   1   7   1   17348   165 58
29  0   2   16  2007    9   26  2006    4   17070   17213   4   1   7   1   16983   143 230
30  0   10  13  2008    5   14  2007    4   17300   17818   3   1   7   1   17348   518 470
31  0   12  22  2007    8   13  2007    3   17391   17522   4   1   7   1   17348   131 174
32  0   4   13  2008    9   7   2007    3   17416   17635   4   1   7   1   17348   219 287
33  0   12  30  2007    7   9   2007    3   17356   17530   3   0   7   1   17348   174 182
34  0   12  1   2006    7   14  2006    4   16996   17136   99  1   7   1   16983   140 153
35  0   2   7   2010    9   15  2007    3   17424   18300   3   1   7   1   17348   876 952
36  0   4   29  2008    6   29  2007    3   17346   17651   3   1   7   1   17348   305 303
37  0   12  4   2007    12  1   2007    3   17501   17504   4   1   7   1   17348   3   156
38  0   9   18  2007    7   16  2007    3   17363   17427   4   1   7   1   17348   64  79
39  0   12  2   2007    10  18  2007    3   17457   17502   3   1   7   1   17348   45  154
40  0   6   23  2008    12  3   2007    4   17503   17706   3   1   7   1   17348   203 358
41  0   8   23  2008    2   4   2008    2   17566   17767   4   1   7   1   17714   201 53
42  0   6   6   2009    4   14  2008    3   17636   18054   4   1   7   1   17714   418 340
43  0   8   21  2011    5   16  2008    3   17668   18860   3   1   7   1   17714   1192    1146
44  0   1   14  2009    9   24  2008    4   17799   17911   4   1   7   1   17714   112 197
45  0   7   9   2009    4   22  2009    4   18009   18087   3   1   7   1   18079   78  8
46  0   9   28  2010    3   17  2009    2   17973   18533   2   0   7   1   18079   560 454
47  0   10  6   2009    4   30  2009    3   18017   18176   4   0   7   1   18079   159 97
48  0   7   10  2009    5   29  2009    4   18046   18088   3   0   7   1   18079   42  9
49  0   8   9   2010    6   4   2009    3   18052   18483   3   1   7   1   18079   431 404
50  0   11  25  2009    7   10  2009    3   18088   18226   4   1   7   1   18079   138 147
51  0   9   16  2009    5   22  2009    4   18039   18156   3   0   7   1   18079   117 77
52  0   4   28  2010    8   21  2009    3   18130   18380   4   1   7   1   18079   250 301
53  0   1   30  2010    9   15  2009    3   18155   18292   2   1   7   1   18079   137 213
54  0   1   21  2010    9   2   2009    3   18142   18283   3   1   7   1   18079   141 204
55  0   10  18  2010    10  9   2009    3   18179   18553   3   0   7   1   18079   374 474
56  0   2   7   2010    10  24  2009    3   18194   18300   4   1   7   1   18079   106 221
57  0   1   26  2010    10  6   2009    3   18176   18288   4   1   7   1   18079   112 209
58  0   2   24  2010    11  19  2009    2   18220   18317   4   1   7   1   18079   97  238
59  0   12  28  2012    11  17  2009    3   18218   19355   4   1   7   1   18079   1137    1276
60  0   6   14  2010    12  2   2009    3   18233   18427   4   1   7   1   18079   194 348
61  0   3   16  2012    11  16  2009    4   18217   19068   3   1   7   1   18079   851 989
62  0   3   20  2010    7   30  2009    3   18108   18341   3   1   7   1   18079   233 262
63  0   5   10  2011    2   23  2010    2   18316   18757   4   1   7   1   18444   441 313
64  0   6   8   2012    3   12  2010    2   18333   19152   4   1   7   1   18444   819 708
65  0   7   10  2010    5   6   2010    4   18388   18453   3   1   7   1   18444   65  9
66  0   7   11  2010    5   25  2010    3   18407   18454   3   1   7   1   18444   47  10
67  0   3   7   2011    7   30  2010    3   18473   18693   4   1   7   1   18444   220 249
68  0   2   7   2012    7   18  2010    2   18461   19030   4   1   7   1   18444   569 586
69  0   12  18  2010    9   3   2010    3   18508   18614   2   0   7   1   18444   106 170
70  0   11  24  2013    11  9   2010    4   18575   19686   3   1   7   1   18444   1111    1242
71  0   4   30  2012    11  12  2010    3   18578   19113   3   1   7   1   18444   535 669
72  0   2   21  2012    11  12  2010    3   18578   19044   3   1   7   1   18444   466 600
73  0   6   12  2011    12  14  2010    3   18610   18790   4   1   7   1   18444   180 346
74  0   7   17  2011    12  20  2010    4   18616   18825   3   1   7   1   18444   209 381
75  0   7   4   2013    11  3   2010    3   18569   19543   3   1   7   1   18444   974 1099
76  0   12  22  2012    10  22  2010    3   18557   19349   3   1   7   1   18444   792 905
77  0   11  28  2013    1   25  2011    3   18652   19690   4   1   7   1   18809   1038    881
78  0   12  27  2011    6   3   2011    3   18781   18988   3   0   7   1   18809   207 179
79  0   12  5   2011    5   2   2011    4   18749   18966   3   1   7   1   18809   217 157
80  0   7   21  2011    5   23  2011    3   18770   18829   4   1   7   1   18809   59  20
81  0   2   27  2014    7   13  2011    3   18821   19781   3   0   7   1   18809   960 972
82  0   12  22  2011    10  12  2011    4   18912   18983   4   1   7   1   18809   71  174
83  0   2   1   2012    9   10  2011    4   18880   19024   4   1   7   1   18809   144 215
84  0   7   14  2012    9   6   2011    4   18876   19188   3   1   7   1   18809   312 379
85  0   12  2   2012    12  9   2011    4   18970   19329   3   1   7   1   18809   359 520
86  0   10  29  2012    12  5   2011    2   18966   19295   4   1   7   1   18809   329 486
87  0   7   10  2015    11  17  2011    1   18948   20279   4   1   7   1   18809   1331    1470
88  0   3   19  2012    12  28  2011    2   18989   19071   4   1   7   1   18809   82  262
89  0   11  16  2012    1   26  2012    3   19018   19313   2   0   7   1   19175   295 138
90  0   2   8   2014    1   5   2012    4   18997   19762   3   1   7   1   19175   765 587
91  0   8   27  2013    2   21  2012    3   19044   19597   4   1   7   1   19175   553 422
92  0   5   2   2013    2   20  2012    3   19043   19480   3   1   7   1   19175   437 305
93  0   2   12  2013    4   13  2012    4   19096   19401   3   1   7   1   19175   305 226
94  0   7   16  2013    4   16  2012    3   19099   19555   2   1   7   1   19175   456 380
95  0   3   4   2013    5   10  2012    3   19123   19421   4   1   7   1   19175   298 246
96  0   7   23  2012    5   14  2012    3   19127   19197   3   1   7   1   19175   70  22
97  0   7   6   2012    5   31  2012    3   19144   19180   4   1   7   1   19175   36  5
98  0   12  22  2012    6   9   2012    2   19153   19349   3   1   7   1   19175   196 174
99  0   6   19  2013    6   28  2012    3   19172   19528   3   1   7   1   19175   356 353
100 0   6   5   2013    8   14  2012    4   19219   19514   3   1   7   1   19175   295 339
101 0   11  5   2012    10  24  2012    4   19290   19302   3   1   7   1   19175   12  127
102 0   8   1   2013    11  9   2012    2   19306   19571   4   1   7   1   19175   265 396
103 0   8   14  2013    11  30  2012    4   19327   19584   4   1   7   1   19175   257 409
104 0   8   3   2012    4   17  2012    3   19100   19208   4   1   7   1   19175   108 33
105 0   3   19  2013    11  5   2012    4   19302   19436   4   1   7   1   19175   134 261
106 0   7   31  2014    11  26  2012    4   19323   19935   4   1   7   1   19175   612 760
107 0   8   18  2014    12  21  2012    3   19348   19953   4   1   7   1   19175   605 778
108 0   11  22  2013    1   10  2013    3   19368   19684   4   1   7   1   19540   316 144
109 0   11  3   2014    1   17  2013    4   19375   20030   3   1   7   1   19540   655 490
110 0   9   14  2013    1   23  2013    4   19381   19615   3   1   7   1   19540   234 75
111 0   12  6   2013    1   30  2013    2   19388   19698   3   1   7   1   19540   310 158
112 0   2   17  2014    3   8   2013    1   19425   19771   4   1   7   1   19540   346 231
113 0   3   4   2014    7   18  2013    2   19557   19786   4   1   7   1   19540   229 246
114 0   8   2   2015    6   11  2013    3   19520   20302   3   1   7   1   19540   782 762
115 0   3   22  2014    10  27  2009    3   18197   19804   2   1   7   1   18079   1607    1725
116 0   11  15  2013    9   26  2013    4   19627   19677   99  1   7   1   19540   50  137
117 0   5   29  2014    11  13  2013    2   19675   19872   4   1   7   1   19540   197 332
118 0   11  6   2014    9   5   2013    3   19606   20033   3   1   7   1   19540   427 493
119 0   7   11  2012    5   10  2012    4   19123   19185   4   1   7   1   19175   62  10
120 0   9   28  2016    12  4   2012    4   19331   20725   3   1   7   1   19175   1394    1550
121 0   8   31  2014    12  21  2013    3   19713   19966   4   1   7   1   19540   253 426
122 0   5   5   2016    1   17  2014    2   19740   20579   4   1   7   1   19905   839 674
123 0   8   1   2015    3   3   2014    2   19785   20301   3   1   7   1   19905   516 396
124 0   11  29  2014    5   16  2014    4   19859   20056   3   1   7   1   19905   197 151
125 0   6   11  2015    7   2   2014    2   19906   20250   3   1   7   1   19905   344 345
126 0   7   18  2015    9   24  2014    3   19990   20287   4   1   7   1   19905   297 382
127 0   7   24  2016    10  30  2014    3   20026   20659   4   1   7   1   19905   633 754
128 0   7   5   2015    12  9   2014    3   20066   20274   4   1   7   1   19905   208 369
129 0   11  19  2015    2   3   2015    1   20122   20411   3   1   7   1   20270   289 141
130 0   10  31  2015    1   8   2015    2   20096   20392   4   1   7   1   20270   296 122
131 0   3   15  2016    3   26  2015    2   20173   20528   4   1   7   1   20270   355 258
132 0   7   4   2015    3   10  2015    3   20157   20273   3   1   7   1   20270   116 3
133 0   8   9   2015    3   17  2015    2   20164   20309   4   1   7   1   20270   145 39
134 0   12  8   2016    5   22  2015    3   20230   20796   2   1   7   1   20270   566 526
135 0   8   15  2016    5   27  2015    4   20235   20681   4   1   7   1   20270   446 411
136 0   7   11  2015    5   8   2015    3   20216   20280   4   1   7   1   20270   64  10
137 0   10  19  2016    7   1   2015    4   20270   20746   3   0   7   1   20270   476 476
138 0   10  28  2015    9   1   2015    3   20332   20389   4   1   7   1   20270   57  119
139 0   2   14  2016    8   6   2015    3   20306   20498   4   1   7   1   20270   192 228
140 0   12  21  2016    11  17  2015    3   20409   20809   4   1   7   1   20270   400 539
141 0   9   12  2016    10  16  2015    3   20377   20709   3   1   7   1   20270   332 439
142 0   4   26  2016    3   2   2015    4   20149   20570   99  0   7   1   20270   421 300
143 0   7   28  2016    8   25  2015    2   20325   20663   3   1   7   1   20270   338 393
;
RUN;

*sort for demo;
proc sort data=have1; 
by stage;
run;


%macro surveyLoop(sample=);

*calculate sample rate;
%let sampRate = %sysevalf(&sample./100);

*create sample;
ods select none;
proc surveyselect data=have1 
                   samprate=&sampRate. 
                   reps=100 
                   out=_sample 
                   seed=995
                   outall;
strata stage;*ensures stage is present for all values - not needed in prod;
run;

*set duration to values if selected;
data _sample;
set _sample;

*assign values as needed;
if selected = 1 then duration = duration_missing;
else duration=duration_no_missing;
run;

*sort for modeling procedures next;
proc sort data=_sample;
by replicate;
run;


/*SURVIVAL ANALYSIS ON THE ACTUAL DATA WITH NO MISSING*/
ods output parameterEstimates = PE_MISS;
PROC PHREG DATA = _sample; 
by replicate;
CLASS STAGE(REF='99') AGEGRP(REF='1')/PARAM=REF; 
MODEL duration_no_missing*DEATH(0) = AGEGRP STAGE/RL EVENTCODE=1; 
*BASELINE OUT = ESTIMATE SURVIVAL = SURVIVAL; 
*output out=resOut; 
RUN;


ods output parameterEstimates = PE_ALL;
/*simulate duration_missing for the 0-100% of the data*/
PROC PHREG DATA = _sample ; 
by replicate;
CLASS STAGE(REF='99') AGEGRP(REF='1')/PARAM=REF; 
MODEL duration*DEATH(0) = AGEGRP STAGE/RL EVENTCODE=1; 
RUN;


*combines results and labels estimates;
data PE_COMBO;
set pe_miss (in=t1) pe_all (in=t2);
length estType $15.;
PCT_MISS = &sample/100;
if t1 then estType = 'Missing';
else estType = 'Full Sample';
run;

*omit if no replicates;
*if replicates calculates average of HR + STDERR for bootstrap approach;
proc means data=pe_combo NWAY N MEAN STD STDERR;
class estType parameter classval0 pct_miss;
var estimate hazardratio;
ods output summary = pe_summary;
run;
ods select all;

*append to main data sets;
proc append base=results_combined data=pe_combo force;
run;

proc append base=results_summary data=pe_summary force;
run;

*clean up after;
proc datasets lib=work nodetails nolist;
delete pe_: _sample;
run;quit;

%let sampRate =;

%mend;


*call macro for different sizes;
%surveyLoop(sample=50);
%surveyLoop(sample=60);
%surveyLoop(sample=70);

Reeza · Posted 06-10-2019 10:35 PM

I've turned off most of the output and graphs to ensure this can run quickly. On my computer, albeit the small sample size, but it's running in pretty much under a second so should scale well, especially since I'm cleaning up the temp data sets at the end.

Cruise · Posted 06-19-2019 10:12 PM

@Reeza

Where and how results_combined results_summary datasets are created? I thought physical datasets are required after base= in the proc append. do you mind to explain?

Ready to join fellow brilliant minds for the SAS Hackathon?

Classroom Training Available!