How to speed up a program ?

[Hugo_B]

Hello I built a program that perform two nested loop in a pretty large table; the program takes a lot of time to be run.

At some point SAS EG alerted me that the log was to big, and that the log will not be printed. After the message appeared the program started to run faster.

Can I speed up my program by not printing the log ? If yes was is the option to do that ?

Is there any other way to speed up a program?

Regards

[PaigeMiller]

You can use PROC PRINTTO to send the log to a file. Here is an example of doing this if you are using Windows:

proc printto log = "c:\data\users\pmiller\temp.log" new; run;
/* Your SAS code goes here */
proc printto; run;

I don't know if this will speed things up enough for you, but you won't get the message that the log window is full.

There are other reasons why your code might take a long time, and the two nested loops may or may not be part of the problem. If you can replace the loops with BY statements, that would speed things up a lot. (But sometimes that is not possible)

In addition, you should examine the log to determine which steps are taking up the most time, and see if there are ways to speed up that particular step.

[VDD]

this code helps me a lot with logging.

ods html close; 
DM log "OUT;CLEAR;LOG;CLEAR;" log continue ;
DM log 'next results; clear; cancel;' whostedit continue ;
ods html newfile=none;

[Hugo_B]

Hello
Can you explain what it does ?
Thank you

[PaigeMiller]

Another thing that can speed up a program (not all programs, but many)

ods graphics off;

This assumes that you don't need ODS GRAPHICS in your program.

[andreas_lds · Accepted solution]

There could be techniques to speed up your program, but without seeing the code, it is difficult to suggest something useful.

[smijoss1]

1) are you using a lot of put statements ? (i generally comment most of them out when promoting)

2) is your Large table a SAS Table OR residing on some database server like oracle. if you do a federated join your performance will be bad.

3) if your dataset is on SAS and program is calculation intensive, then you must check out Proc DS2 for multi-thread processing.

[Hugo_B]

Here is a simplified version of what I do :

%MACRO MacroProgram;

%let i=1;

data TableWant;
Lenght Variable1 $200;
run;

%DO %WHILE (&i<150);

data _null_;
set TableA;
if _N_=&i then 
call symput('MacroVariable',SomeVariableFromTableA)
run;

data TEMP;
VariableTableTemp=&Macrovariable;
run;

Proc Append base=TableWant data=TEMP;

%let i=%eval(&i+1);

%mend;

There is a 45000+ notes in the log saying:

-that some variables are uninitialized;

-some numeric values have been converted ro characters values;

Maybe this is the reason why it takes time.

[smijoss1]

Are you checking if the variable exists on your source dataset. 

the uninitialized is because the variable does not exist.

and the numeric to char warning is because  Symput is converting it to char.

is your TableA that is pretty large ?  ( i feel using IF is inefficient there thus asking)

Looking at you code are you trying to fetch the first value of some column for the first 150 rows ? (OR a range of rows ?)

[Hugo_B]

What can I change in this code to avoid unitialized variables :

DATA TEMP;
LENGTH VARIABLE1-VARIABLE10 $200;
OUTPUT;
RUN;

By doing this i want to "initialize" the TEMP TABLE  before adding values later in the macro program, but I feel that it's wrong.

What do you think ?

[smijoss1]

I am assuming you need a table without any rows for initialization 

DATA TEMP;
LENGTH VARIABLE1-VARIABLE10 $200;

call missing(of VARIABLE1-VARIABLE10);

STOP;
RUN;

I still feel the log isnt reason for your performance. but give it a shot.

[ChrisNZ]

DATA TEMP;
LENGTH VARIABLE1-VARIABLE10 $200;
OUTPUT;
RUN;

is normally better coded as

data TEMP;
  if 0 then set SOURCETAB (keep=VARIABLE1-VARIABLE10);
run;

as this copies the original structure.

Also, $200 variables create long records, and SAS is not efficient dealing with these. Shorten the variables if you can. And compress the tables.

[Hugo_B]

%MACRO MYPROGRAM;

%LET I=1;

%DO %WHILE (&i<200)

DATA _NULL_;
SET TABLEA;
IF _N_=&i THEN CALL SYMPUT('MACROVARIABLE',VarThatINeed);
RUN;

DATA TEMP;
SET TEMP;
VARIABLE=SYMGET('MACROVARIABLE');
RUN;

PROC APPEND BASE=TABLEWANT DATA=TEMP;

%LET &I=%EVAL(&i+1)
%MEND;

I am pretty sure the IF in the data step is not efficient, is there a way to point directly to i th observation without using " if _n_=&i " ?

[smijoss1]

pointing to Nth observation - you can use POINTOBS

But going back to my earlier question - are you just trying to fetch the values of certain columns for first 150 or 200 rows 
(or a range like between row# 100 and 200 ) 

[Hugo_B]

All rows, but one row at a time, one single row at each stage of the loop and using 5 columns/variables for every row.

I am using SAS EG GRID running on server, can it be a reason for slowness ?