The brilliance of Custom Functions in Intelligent Decisioning
Introduction
I was recently presented with the following problem. The use case required a business rule to assess if a list of procedure codes (input_string) were present in an existing reference list of procedure codes (delimited_string_list) and return a count of how many matches (matches_found) of the procedure codes from the input list were present in the reference list. Both lists were provided as comma separated list of string values. The order of the procedure codes in the input list is completely random. By way of a simplified visual the use case is presented as follows:
Problem
Most functions available in the rule expression builder and SAS in general to allow string manipulation or searches, for example SUBSTR, FIND or FINDC, allows in part to achieve the requirement, however the problem comes with searching for multiple string values in a unpredictable order within a string of say delimited values.
For example the FINDC function in its simplest form : FINDC( 'ABC123, KEL567, POC678, LOP654','KEL567',1) will return a value of 9, indicating that the string 'KEL567' was found in position 9 of the primary string. I could use this output in by business rule to indicate that the code 'KEL567' was found in the reference list.
The problem though is that this function will fail if I was looking for more than one procedure code in the reference list for example : FINDC( 'ABC123, KEL567, POC678, LOP654','KEL567,ABC123',1). This function will return 0 even though both procedure codes 'KEL567', and 'ABC123' exists in the reference list.
As far as I know there is not a single function which will allow me to achieve this using an "out of the box" SAS function in a rule expression. If there is please comment in the comment section below.
The Solution
The Custom Function Definition
Enter the custom function. You can use custom functions to perform actions that are not available with the standard functions that SAS provides. Custom functions also enable you to encapsulate and reuse business logic. You can use custom functions in rule sets and in DS2 code files.
The beauty of custom functions are that when you add a function category and custom function to SAS Intelligent Decisioning, SAS Intelligent Decisioning adds the category and function to the list of functions in the rule set expression editor. Custom categories appear as subcategories under the category Custom. More on this later.
You start the process of creating a new custom function by clicking New Custom Function in the Custom Function category view within Intelligent Decisioning. The UI allows you to define the custom function code, define properties and assign input and output variables. The remainder of this article assumes a level of programming experience and wont go into the details of the code logic.
The custom function code definition starts with a method statement. Using the method statement you assign a name to the function, which you will use in the business rule expression editor to reference your new custom function. In addition you would pass input variables to the function and you would expect the function to return values. In the example below the custom function is called "cf_string_parse" and it requires two input variables i.e. delimited_string_list (reference list) and input_string (list of procedure codes to match against the reference list). The function will return a single value of type integer. This value is assigned by the maches_found variable later on in the code.
method cf_string_parse(
varchar(200) delimited_string_list,
varchar(100) input_string)
returns integer;
Any variables used within the code has to be declared else the system will generate errors. In addition default values are assigned to the variables.
/* declare variables */
dcl varchar(100) c;
dcl varchar(1) delimiter;
dcl integer start_pos end_pos temp_pos found_code k found matches_found;
/* assign default values */
k=0;
c='';
start_pos = 1;
end_pos=0;
temp_pos = 0;
found_code = 0;
matches_found=0;
delimiter =',';
I used the COMPRESS function to remove any spaces in either lists to make sure the process is not complicated by unwanted spaces.
/* remove all spaces from the delimited string list & input_string */
delimited_string_list = COMPRESS(delimited_string_list) ;
input_string = COMPRESS(input_string) ;
Consider the input list with assigned string "KEL567,ABC123". This list has two procedure codes which may or may not be found in the reference list. A delimiter (comma) is used to separate each procedure code in the list. The code example below uses the "do until" function to allow the logic to repeat certain actions until, in this case, the end of the input string is reached. The code allows therefore to find the first procedure code "KEL567", search for it in the reference list using the FINDC function, then look for the second procedure code "ABC123" in the reference list. The "do until" loop will terminate once the end of the input list is reached or rather in this case when no more "delimiters" can be found. Each time a match is found the mached_found variable is incremented by one to keep a count.
/* loop through the input_string values delimited by a set delimiter */
/* find any matching items in the delimited_string_list */
do until (k=0);
k = FINDC(input_string,delimiter,k+1) ;
if k=0 then do;
c=strip(substr(input_string, start_pos));
found_code = FINDW(delimited_string_list,c,delimiter,1);
if found_code > 0 then
do;
matches_found = matches_found + 1;
end;
end;
else do;
end_pos = k - 1;
c=strip(substr(input_string, start_pos, end_pos-temp_pos));
temp_pos = k;
start_pos = k+1;
/* Find the code in the delimited_string_list */
found_code = FINDW(delimited_string_list,c,delimiter,1);
if found_code > 0 then
do;
matches_found = matches_found + 1;
end;
end;
end;
Once the end of the input list is reached, the variable matched_found is returned to the method invocation as a return variable for the custom function.
EXIT:
return matches_found;
end;
Note that the custom function also requires you to set the input and output variables in order for you to map the variables to rule variables later on. This is done on the variables tab of the custom function UI. You will notice that it is not required to map the return variable (matched_count) as an output variable. By virtue of the custom function, the function returns an output by default of the type specified (integer) to the rule expression where it is called from.
Using the "Validate" button in the UI, the code will be validated and any errors in the code will be reported.
Using the Custom Function in a Rule
Now we can use our custom function (CF_STRING_PARSE) in a rule expression. The process to use the custom function is similar to using any other function available in the rule expression editor. To test our new custom function we can define a new Assignment rule. In the expression editor, the new custom function is shown in the "Custom" section, below the standard functions. Double-click on the new function and the new function will be added to the editor window on the right. Drag an drop the input variables of the rule to the function variable place holders. The functio nwill return the number of matches in the "matched_found" variable.
Once the expression is built the assignment rule will look similar to the following :
We can now run a scoring test to test if the new custom function is producing the desired output:
The scoring test produces the following result:
In Summary
You can use custom functions to perform actions that are not available with the standard functions that SAS provides. Custom functions also enable you to encapsulate and reuse business logic. You can use custom functions in rule sets and in DS2 code files.
Complete example code :
method cf_string_parse(
varchar(200) delimited_string_list,
varchar(100) input_string)
returns integer;
/* declare variables */
dcl varchar(100) c;
dcl varchar(1) delimiter;
dcl integer start_pos end_pos temp_pos found_code k found matches_found;
/* assign default values */
k=0;
c='';
start_pos = 1;
end_pos=0;
temp_pos = 0;
found_code = 0;
matches_found=0;
delimiter =',';
/* remove all spaces from the delimited string list & input_string */
delimited_string_list = COMPRESS(delimited_string_list) ;
input_string = COMPRESS(input_string) ;
/* loop through the input_string values delimited by a set delimiter */
/* find any matching items in the delimited_string_list */
do until (k=0);
k = FINDC(input_string,delimiter,k+1) ;
if k=0 then do;
c=strip(substr(input_string, start_pos));
found_code = FINDW(delimited_string_list,c,delimiter,1);
if found_code > 0 then
do;
matches_found = matches_found + 1;
end;
end;
else do;
end_pos = k - 1;
c=strip(substr(input_string, start_pos, end_pos-temp_pos));
temp_pos = k;
start_pos = k+1;
/* Find the code in the delimited_string_list */
found_code = FINDW(delimited_string_list,c,delimiter,1);
if found_code > 0 then
do;
matches_found = matches_found + 1;
end;
end;
end;
EXIT:
return matches_found;
end;
