Well, i added an options sumsize=max; That appears to have drastically improved things. This one just ran in 28 minutes instead of 3 hours. I also added a bunch of put statements to help track timing. It doesn't appear that the time it takes is used in reading them. Maybe in building the index at this point. MLOGIC(YEARS): Beginning execution. SYMBOLGEN: Macro variable BEGYR resolves to 2011 SYMBOLGEN: Macro variable ENDYR resolves to 2011 MLOGIC(YEARS): %DO loop beginning; index variable YEAR; start value is 2011; stop value is 2011; by value is 1. MLOGIC(YEARS): %PUT all claims set start %sysfunc(time(),timeampm.) all claims set start 12:24:52 PM SYMBOLGEN: Macro variable YEAR resolves to 2011 MPRINT(YEARS): data allclaims_2011 (index=(key=(usrds_id hcfasaf clm_from clm_thru costtype))); MLOGIC(YEARS): %PUT all claims read op %sysfunc(time(),timeampm.) all claims read op 12:24:52 PM SYMBOLGEN: Macro variable YEAR resolves to 2011 MLOGIC(YEARS): %PUT all claims read hh %sysfunc(time(),timeampm.) all claims read hh 12:24:52 PM SYMBOLGEN: Macro variable YEAR resolves to 2011 MLOGIC(YEARS): %PUT all claims read hs %sysfunc(time(),timeampm.) all claims read hs 12:24:52 PM SYMBOLGEN: Macro variable YEAR resolves to 2011 MLOGIC(YEARS): %PUT all claims read sn %sysfunc(time(),timeampm.) all claims read sn 12:24:52 PM SYMBOLGEN: Macro variable YEAR resolves to 2011 MLOGIC(YEARS): %PUT all claims read ip %sysfunc(time(),timeampm.) all claims read ip 12:24:52 PM SYMBOLGEN: Macro variable YEAR resolves to 2011 MLOGIC(YEARS): %PUT all claims read ps %sysfunc(time(),timeampm.) all claims read ps 12:24:52 PM SYMBOLGEN: Macro variable YEAR resolves to 2011 MLOGIC(YEARS): %PUT all claims read pde %sysfunc(time(),timeampm.) all claims read pde 12:24:52 PM SYMBOLGEN: Macro variable YEAR resolves to 2011 MLOGIC(YEARS): %IF condition &year. >= 2006 is TRUE SYMBOLGEN: Macro variable YEAR resolves to 2011 MPRINT(YEARS): set junk.op_2011 junk.hh_2011 junk.hs_2011 junk.sn_2011 clms.ip_2011 (drop=mod1 dm) junk.ps_2011 junk.partd_2011 ; MPRINT(YEARS): run; NOTE: There were 71739902 observations read from the data set JUNK.OP_2011. NOTE: There were 256100 observations read from the data set JUNK.HH_2011. NOTE: There were 52816 observations read from the data set JUNK.HS_2011. NOTE: There were 209802 observations read from the data set JUNK.SN_2011. NOTE: There were 1164526 observations read from the data set CLMS.IP_2011. NOTE: There were 58473794 observations read from the data set JUNK.PS_2011. NOTE: There were 18251360 observations read from the data set JUNK.PARTD_2011. NOTE: The data set WORK.ALLCLAIMS_2011 has 150148300 observations and 48 variables. NOTE: Compressing data set WORK.ALLCLAIMS_2011 decreased size by 62.12 percent. Compressed is 1387318 pages; un-compressed would require 3662157 pages. NOTE: DATA statement used (Total process time): real time 28:00.88 user cpu time 7:55.95 system cpu time 2:58.30 Memory 67394k OS Memory 71684k Timestamp 6/26/2016 12:52:52 PM all claims done set 12:52:53 PM
... View more