Scoring Logic and how to process faster

[gra_in_aus]

Hi All,

I have a scoring file that looks like IML code that is taking long run time to run.  I have tried loading the data into memory and the performance does not improve.  I have also tried to use PROC DS2 (SAS 9.4m3) without success as it fails to process the GOTO and colon ':' references.  BUFSIZE, IBUFSIZE, UBUFSIZE is 64k and BUFNO is set to 10.  I have plenty of memory at 120GB with 16 vcpus.

Example scoring code:

if tqge > .z then goto N19_2;
else goto T19_8;

T19_8:
response = 2;
goto D19;

N19_2:
if tqge < 999 then goto N19_3;
else goto N19_5;

N19_3:
if statex in ('ACT','Gold Coast','NT','North QLD','South QLD','TAS','WA'
) then goto T19_1;
else if statex in ('NSW','SA','VIC'
) then goto N19_4;
else goto T19_1;

T19_1:
response = -0.05;
goto D19;

N19_4:
if agex <= .z then goto T19_3;
if agex < 50 then goto T19_2;
else goto T19_3;

Any help would be greatly appreciated.

G

[SASKiwi]

One reliable way of speeding this up would be to use parallel processing. For example if you split your input data into say quarters then had four identical SAS jobs doing a quarter each then I'm sure it would run faster.

One of the easiest ways to do parallel processing is to use SAS/CONNECT to create additional SAS sessions on the same SAS server and run them asynchronously (that is in parallel) then have a master SAS job wait four all of the child sessions to finish to combine the final results. Even if you don't have SAS/CONNECT you could do the same thing by running multiple, simultaneous SAS batch jobs.

BTW your code snippet looks more like SAS DATA step code to me. It would be helpful to see the complete step. 

[IanWakeling]

There is no IN operator in IML, so I agree that this looks like DATA step code.

[gra_in_aus]

Thanks for the suggestion,  I have thought about parallel processing the code, however it is entwined into a lot of macro code and SAS/CONNECT is not available on the platform I am using :(.  That's why I want to know if there was an alternate solution either by configuration settings, DS2 or other means.

Many thanks

G

[Rick_SAS]

This is not IML code. I assume it is the result from the CODE statement of an analytical  procedure. What procedure generated the code? That information will help determine what option you have to speed it up.

One general idea: string processing is more expensive than numerical processing. If you can recode the data to use numerical values (for example, 0 and 1 instead of "No" and "Yes"), then the processing time can improve.

Another idea: Look at using the STORE statement and PROC PLM to do the scoring rather than the CODE statement. Whether that is possible depends on the procedure that generated this code.

[Damien_Mather]

Hi Rick.

I think I can answer this 'on behalf' of the original posting community member, in case it takes too long for them to reply to your (as usual) very helpful post. I teach the SAS Advanced Business Analytics (ABA) course, not heavily disguised at all as an 8 north american university credit equivalent postgraduate marketing course at my (southern hemisphere) university. I recognized to scoring code immediately. It has the distinctive structural signature of scoring code produced by SAS Enterprise Miner, hence the additional comment in the original post about the scoring code being heavily entwined in SAS maco code. SAS Enterprise Miner makes heavy use of macro code to produce the scoring code when requested via the 'score code' property of the model comparison node in SAS Enterprise Miner. SAS EM scoring code is designed to be cut and pasted into either a minimal pair of SAS data step wrapper statements, or similar SAS DS2 step wrapper statements, or a minimal set of wrapper statement in maximal number of other data processing languages that share the SAS data step syntax element subset employed by the SAS Enterprise Miner model node scoring function.

I hope that observation helps!