Dear all, I have a macro (%foreign_macro) that needs to process a pretty big sas data set containing time series data ("very_large"). Unfortunately, with each execution the macro can only consider one specific day in the input data set (here: &input_data.=very_large). The macro is pretty fast for dates at the beginning of the data set, but the longer it has to search through the data, the longer it takes to execute. After the 80th iteration (and >300 to go), I had to stop SAS because it already took the macro several minutes to execute. In the example below, the macro %my_macro executes the aforementioned macro %foreign_macro for each of the dates given in the macro variable &dates. Uncommenting the code reveals the idea that I had for lowering the execution time. First, a duplicate of "very_large" called "_very_large" is made for use in further steps (disk space is not a restriction... yet). Second, before executing %foreign_macro, data that is no longer needed gets removed from the (duplicate) data set. This way, the macro doesn't have to go over this data time and time again with each execution. Unfortunately, when using modify & remove, it seems that data isn't actually (physically) removed. The execution time still increased with each iteration, and while the program was running "_very_large" never decreased in file size (it remained the exact same size as "very_large"). My first question would be if there's another way to remove the first observations of a data set (other than using modify & remove)? The second question is if there's a better solution for my problem (other than to rewrite %foreign_macro)? Thank you very much for your time. Any help is appreciated! Data very_large; Attrib Date Length=4 Informat=date11. Format=date11.; Attrib Vars Length=3; Input Date Vars; Datalines; 09-JAN-2013 10 09-JAN-2013 20 09-JAN-2013 10 10-JAN-2013 20 11-JAN-2013 10 11-JAN-2013 20 Run; %Macro foreign_macro( input_data=, output_data=, date= ); Data &output_data.; Set &input_data.; If Date>&date. Then Stop; If Date=&date. Then Do; Vars=Vars/2; Output; End; Run; %MEND; %Macro my_macro(input_data=,dates=); %Let dates_num=%eval(%sysfunc(count(%str(&dates.),%str(,)))+1); /* * FIRST: work with a duplicate of very_large ; Data _&input_data.; Set &input_data.; Run; %Let input_data=_&input_data.; */ %DO i=1 %TO &dates_num.; /* * SECOND: remove data from the duplicate that is no longer needed ; Data &input_data.; Modify &input_data.; If Date>=%Scan(&dates,&i.,%Quote(,)) Then Stop; Remove; Run; */ %foreign_macro( input_data=&input_data., output_data=output&i., date=%Scan(&dates.,&i.,%Quote(,)) ); %End; %Mend; %my_macro( input_data=very_large, dates=%str("09-JAN-2013"d,"11-JAN-2013"d) ); Nachricht wurde geändert durch: Jörg Honnacker
... View more