BookmarkSubscribeRSS Feed

The brilliance of Custom Functions in Intelligent Decisioning

Started ‎07-21-2023 by
Modified ‎07-21-2023 by
Views 376

Introduction

 

I was recently presented with the following problem. The use case required a business rule to assess if a list of procedure codes (input_string) were present in an existing reference list of procedure codes  (delimited_string_list) and return a count of how many matches (matches_found) of the procedure codes from the input list were present in the reference list. Both lists were provided as comma separated list of string values. The order of the procedure codes in the input list is completely random. By way of a simplified visual the use case is presented as follows:

 

Gerrit_3-1689906637456.png

 

Problem 

 

Most functions available in the rule expression builder and SAS in general to allow string manipulation or searches, for example SUBSTR, FIND or FINDC, allows in part to achieve the requirement, however the problem comes with searching for multiple string values in a unpredictable order within a string of say delimited values. 

 

For example the FINDC function  in its simplest form : FINDC( 'ABC123, KEL567, POC678, LOP654','KEL567',1) will return a value of 9, indicating that the string 'KEL567' was found in position 9 of the primary string. I could use this output in by business rule to indicate that the code 'KEL567' was found in the reference list.

 

The problem though is that this function will fail if I was looking for more than one procedure code in the reference list for example : FINDC( 'ABC123, KEL567, POC678, LOP654','KEL567,ABC123',1). This function will return 0 even though both procedure codes 'KEL567', and 'ABC123' exists in the reference list. 

 

As far as I know there is not a single function which will allow me to achieve this using an "out of the box" SAS function in a rule expression. If there is please comment in the comment section below.

 

The Solution

 

The Custom Function Definition

 

Enter the custom function. You can use custom functions to perform actions that are not available with the standard functions that SAS provides. Custom functions also enable you to encapsulate and reuse business logic. You can use custom functions in rule sets and in DS2 code files.

 

The beauty of custom functions are that when you add a function category and custom function to SAS Intelligent Decisioning, SAS Intelligent Decisioning adds the category and function to the list of functions in the rule set expression editor. Custom categories appear as subcategories under the category Custom. More on this later.

 

You start the process of creating a new custom function by clicking New Custom Function in the Custom Function category view within Intelligent Decisioning. The UI allows you to define the custom function code, define properties and assign input and output variables. The remainder of this article assumes a level of programming experience and wont go into the details of the code logic. 

 

The custom function code definition starts with a method statement. Using the method statement you assign a name to the function, which you will use in the business rule expression editor to reference your new custom function. In addition you would pass input variables to the function and you would expect the function to return values. In the example below the custom function is called "cf_string_parse" and it requires two input variables i.e. delimited_string_list (reference list) and input_string (list of procedure codes to match against the reference list). The function will return a single value of type integer. This value is assigned by the maches_found variable later on in the code. 

 

method cf_string_parse(
           varchar(200) delimited_string_list,
           varchar(100) input_string) 
		   returns integer; 

 

Any variables used within the code has to be declared else the system will generate errors. In addition default values are assigned to the variables.

 

   /* declare variables */
   dcl varchar(100) c;
   dcl varchar(1) delimiter;
   dcl integer start_pos end_pos temp_pos found_code k found matches_found;

   /* assign default values */ 
   k=0;
   c='';
   start_pos = 1;
   end_pos=0;
   temp_pos = 0;
   found_code = 0;
   matches_found=0;
   delimiter =',';

 

I used the COMPRESS function to remove any spaces in either lists to make sure the process is not complicated by unwanted spaces.

 

   /* remove all spaces from the delimited string list & input_string     */
   delimited_string_list = COMPRESS(delimited_string_list) ;
   input_string = COMPRESS(input_string) ;

 

Consider the input list with assigned string "KEL567,ABC123". This list has two procedure codes which may or may not be found in the reference list. A delimiter (comma) is used to separate each procedure code in the list. The code example below uses the "do until" function to allow the logic to repeat certain actions until, in this case, the end of the input string is reached. The code allows therefore to find the first procedure code "KEL567", search for it in the reference list using the FINDC function, then look for the second procedure code "ABC123" in the reference list. The "do until" loop will terminate once the end of the input list is reached or rather in this case when no more "delimiters" can be found. Each time a match is found the mached_found variable is incremented by one to keep a count.

 

/* loop through the input_string values delimited by a set delimiter */
   /* find any matching items in the delimited_string_list                         */
   do until (k=0);
		k = FINDC(input_string,delimiter,k+1) ;
   		if k=0 then  do;
   	 		c=strip(substr(input_string, start_pos));
   	    	found_code = FINDW(delimited_string_list,c,delimiter,1);
   	    	if found_code > 0 then 
   	    		do;
   	    			matches_found = matches_found + 1;
   	    		end;
 	    end;
     	else do;
      	 	end_pos = k - 1; 
         	c=strip(substr(input_string, start_pos, end_pos-temp_pos));
         	temp_pos = k;
         	start_pos = k+1;
         	/* Find the code in the delimited_string_list                        */
         	found_code = FINDW(delimited_string_list,c,delimiter,1);
         	if found_code > 0 then 
         		do;
         			matches_found = matches_found + 1;
         		end;
      end;
   end;

 

Once the end of the input list is reached, the variable matched_found is returned to the method invocation as a return variable for the custom function. 

 

EXIT:
   return matches_found;
end;

 

Note that the custom function also requires you to set the input and output variables in order for you to map the variables to rule variables later on. This is done on the variables tab of the custom function UI. You will notice that it is not required to map the return variable (matched_count) as an output variable. By virtue of the custom function, the function returns an output by default of the type specified (integer) to the rule expression where it is called from. 

 

Gerrit_0-1689912102968.png

 

Using the "Validate" button in the UI, the code will be validated and any errors in the code will be reported.

 

Using the Custom Function in a Rule

 

Now we can use our custom function (CF_STRING_PARSE) in a rule expression. The process to use the custom function is similar to using any other function available in the rule expression editor. To test our new custom function we can define a new Assignment rule. In the expression editor, the new custom function is shown in the "Custom" section, below the standard functions.  Double-click on the new function and the new function will be added to the editor window on the right. Drag an drop the input variables of the rule to the function variable place holders. The functio nwill return the number of matches in the "matched_found" variable.

 

Gerrit_2-1689912673647.png

 

Once the expression is built the assignment rule will look similar to the following :

 

Gerrit_3-1689912958985.png

 

 We can now run a scoring test to test if the new custom function is producing the desired output:

 

Gerrit_4-1689913180129.png

 

The scoring test produces the following result:

 

Gerrit_5-1689913276658.png

 

In Summary

 

You can use custom functions to perform actions that are not available with the standard functions that SAS provides. Custom functions also enable you to encapsulate and reuse business logic. You can use custom functions in rule sets and in DS2 code files.

 

Complete example code :

 

method cf_string_parse(
           varchar(200) delimited_string_list,
           varchar(100) input_string) 
		   returns integer; 
   
   /* declare variables */
   dcl varchar(100) c;
   dcl varchar(1) delimiter;
   dcl integer start_pos end_pos temp_pos found_code k found matches_found;

   /* assign default values */ 
   k=0;
   c='';
   start_pos = 1;
   end_pos=0;
   temp_pos = 0;
   found_code = 0;
   matches_found=0;
   delimiter =',';
   
   /* remove all spaces from the delimited string list & input_string     */
   delimited_string_list = COMPRESS(delimited_string_list) ;
   input_string = COMPRESS(input_string) ;
   
   /* loop through the input_string values delimited by a set delimiter */
   /* find any matching items in the delimited_string_list                         */
   do until (k=0);
		k = FINDC(input_string,delimiter,k+1) ;
   		if k=0 then  do;
   	 		c=strip(substr(input_string, start_pos));
   	    	found_code = FINDW(delimited_string_list,c,delimiter,1);
   	    	if found_code > 0 then 
   	    		do;
   	    			matches_found = matches_found + 1;
   	    		end;
 	    end;
     	else do;
      	 	end_pos = k - 1; 
         	c=strip(substr(input_string, start_pos, end_pos-temp_pos));
         	temp_pos = k;
         	start_pos = k+1;
         	/* Find the code in the delimited_string_list                        */
         	found_code = FINDW(delimited_string_list,c,delimiter,1);
         	if found_code > 0 then 
         		do;
         			matches_found = matches_found + 1;
         		end;
      end;
   end;
EXIT:
   return matches_found;
end;

 


 


 


 

 

 

Version history
Last update:
‎07-21-2023 12:28 AM
Updated by:
Contributors

sas-innovate-2024.png

Don't miss out on SAS Innovate - Register now for the FREE Livestream!

Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.

 

Register now!

Free course: Data Literacy Essentials

Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning  and boost your career prospects.

Get Started

Article Tags