topic Re: apply user defined format UDF in SAS Viya

apply user defined format UDF

acordes — Tue, 11 Feb 2020 15:19:28 GMT

I have followed the steps to cretae a global scope user-defined-format.

In "Management environment" it shows up correctly.

But now I do not have any clue about applying it correctly.

I want assign the mpgrating format.

79    cas mysession listformats scope=global members;
NOTE: Fmtlib = MYFMTLIB          
         Scope = Global
         Fmtsearch = NA
         Format = mpgrating  
NOTE: Fmtlib = SASSUPPLIEDFORMATS
         Scope = Global
         Fmtsearch = NA
         Format = $obfsusr   
         Format = $shift2txt 
         Format = $shiftxt2id
         Format = shift      
         Format = shifttxt

this doesn't work as I expected because it's not cas but data step.

data public.class ;
format age mpgrating. ;
set sashelp.class;
age=age*3 + rand("normal",5,13);
run;

Should I do it with proc casutil? How?

My ultimative goal is to use this UDF in Visual Analytics.

Re: apply user defined format UDF

BrunoMueller — Wed, 12 Feb 2020 08:06:19 GMT

Your program will work, just a few things to note:

numeric variables with a user defined format will be treated as category
you have to promote the table so it has a global scope and can be seen in Visual Analytics
a global scoped table (promoted) needs to be deleted first so that it can be written again
when running the DATA Step the format has to be available to SAS, or use the NOFMTERR system option

This code sample works for me:

cas sugus sessopts=(caslib="casuser");
cas sugus listformats scope=global members;

libname public cas caslib="public";

options nofmterr;

proc delete data=public.class_format_test;
run;

data public.class_format_test(promote=yes);
  set sashelp.class;
  ageGroup = age;
  format ageGroup class_age_group.;
run;

Re: apply user defined format UDF

acordes — Thu, 13 Feb 2020 09:28:08 GMT

Thank you @BrunoMueller

But it doesn't work.

here comes the log.

1     OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

NOTE: The CAS statement request to update one or more session options for session SUGUS completed.
77    cas sugus listformats scope=global members;
NOTE: Fmtlib = MYFMTLIB          
         Scope = Global
         Fmtsearch = NA
         Format = mpgrating  
NOTE: Fmtlib = SASSUPPLIEDFORMATS
         Scope = Global
         Fmtsearch = NA
         Format = $obfsusr   
         Format = $shift2txt 
         Format = $shiftxt2id
         Format = shift      
         Format = shifttxt   
NOTE: Request to LISTFORMAT completed for session SUGUS.
78    
79    libname public cas caslib="public";
NOTE: Libref PUBLIC was successfully assigned as follows: 
      Engine:        CAS 

81    options nofmterr;
82    
83    /* proc delete data=public.class_format_test; */
84    /* run; */
85    
86    data public.class_format_test;
87      set sashelp.class;
88      ageGroup = age;
89      format ageGroup mpgrating.;
                        ----------
                        484
NOTE 484-185: Format MPGRATING was not found or could not be loaded.
90    run;
NOTE: There were 19 observations read from the data set SASHELP.CLASS.
NOTE: The data set PUBLIC.CLASS_FORMAT_TEST has 19 observations and 6 variables.
NOTE: DATA statement used (Total process time):
      real time           0.01 seconds
      cpu time            0.03 seconds
      
91    
92    OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
105

Re: apply user defined format UDF

BrunoMueller — Thu, 13 Feb 2020 12:25:22 GMT

You are missing two critical elements I mentioned before:

a global scoped table (promoted) needs to be deleted first so that it can be written again
```
proc delete data=public.class_format_test;
run;
```
you have to promote the table so it has a global scope and can be seen in Visual Analytics
```
data public.class_format_test(promote=yes);
```

The NOTE message about the format "NOTE 484-185: Format MPGRATING was not found or could not be loaded." just says it can not be found in the SAS environment where the program is running, it will be found on the CAS server.

Re: apply user defined format UDF

acordes — Mon, 17 Feb 2020 11:12:47 GMT

It still doesn't work.

Format seems ok but there is no way to apply it.

cas accas listfmtranges fmtname=MPGrating;

1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

76 cas accas listfmtranges fmtname=MPGrating;

Format Name Range

MPGRATING LOW-<19=Poor

19-<24=Fair

24-<34=Good

34-HIGH=Excellent

NOTE: Request to LISTFMTRANGES MPGRATING completed for session ACCAS.

78 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

cas accas sessopts=(caslib="casuser");
cas accas listformats scope=global members;

libname public cas caslib="public";

options nofmterr;

proc delete data=public.class_format_test;
run;

data public.class_format_test(promote=yes);
  set sashelp.class;
  ageGroup = age*3+rand("normal",0, 4);
  format ageGroup MPGRATING.;
run;

81 data public.class_format_test(promote=yes);

82 set sashelp.class;

83 ageGroup = age*3+rand("normal",0, 4);

84 format ageGroup MPGRATING.;

----------

484

NOTE 484-185: Format MPGRATING was not found or could not be loaded.

85 run;

NOTE: There were 19 observations read from the data set SASHELP.CLASS.

NOTE: The data set PUBLIC.CLASS_FORMAT_TEST has 19 observations and 6 variables.

NOTE: The data set CLASS_FORMAT_TEST was promoted.

NOTE: DATA statement used (Total process time):

real time 0.01 seconds

cpu time 0.00 seconds

87 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

100

Re: apply user defined format UDF

acordes — Mon, 17 Feb 2020 17:25:15 GMT

And when I open VA this error prompt appears:

com.sas.cas.CASException: The user-defined programming statements could not be parsed. (severity=2 reason=6 statusCode=2710055)
5 ERROR: The user-defined programming statements could not be parsed.
5 ERROR: Invalid FORMAT 'ENGINESIZE.' found.
5 ERROR: Failure opening table 'CARS'
5 ERROR: The action stopped due to errors.
debug=0x887ff837:TKCASTAB_PARSE_ERROR

In SAS Studio the format was applied correctly when I wrote the table to a caslib.

Re: apply user defined format UDF

BrunoMueller — Mon, 17 Feb 2020 20:02:19 GMT

Please run the below code, it will show information about the cas table (similar to what you get with Proc contents). It will also run something similar to "proc freq" using the column with the user defined format. You might need to adjust names for caslib= and name= etc. depending on how you are testing.


proc cas;
  action table.columninfo / table={caslib="public" name="class_format_test"};
  action freqTab.freqTab / table={caslib="public" name="class_format_test"} 
  tabulate={
  	{vars={'ageGroup'}}
  }
  display={ names={"OneWayFreqs"} } 
  ;
run;
quit;

Re: apply user defined format UDF

BrunoMueller — Mon, 17 Feb 2020 20:04:37 GMT

You show log with a table named class_format_test, but the message above is for a different table as well as a different format.

Have you checked in VA with the table class_format_test in cas library public?

Re: apply user defined format UDF

acordes — Tue, 18 Feb 2020 08:59:38 GMT

I'd like to thank you for not giving up with my issue 🙂

Running the above code works in SAS Studio to assign the desired format.

But that's not the final goal! I want the UDF to be applied in Visual Analytics.

And I cannot find a way to achieve this.

There are still many open questions related to this:

How do I delete a format? I had to create a new one, class_age, because it didn't let overwrite the existing class_age_group
How Do I interpret the attached output of "listformats"? Does my UDF require global scope and fmtseach put to YES in order to be found in Visual Analytics? When entering the format catalogues under Manage Environment I can see my UDF

85 cas accas listformats;

NOTE: Fmtlib = FMTLIB1

Scope = Global

Fmtsearch = NA

NOTE: Fmtlib = MYFMTLIB

Scope = Global

Fmtsearch = NA

NOTE: Fmtlib = MYFMTLIB

Scope = Session

Fmtsearch = YES

87 cas accas listformats scope=session members;

NOTE: Fmtlib = MYFMTLIB

Scope = Session

Fmtsearch = YES

Format = mpgrating

cas sugus sessopts=(caslib="casuser");
cas sugus listformats scope=global members;

libname public cas caslib="public";

options nofmterr;

cas sugus dropfmtlib fmtlibname=myFmtLib  /* 2 */
fmtsearchremove;

proc format sessref=sugus casfmtlib="myFmtLib";  /* 4 */
   value class_age
      13  -  HIGH = "old"
      12  -< 13   = "normal"
      LOW -< 12   = "young";
run;


proc delete data=public.class_format_test;
run;

data public.class_format_test(promote=yes);
  set sashelp.class;
  ageGroup = age;
  format ageGroup class_age.;
run;

proc cas;
  action table.columninfo / table={caslib="public" name="class_format_test"};
  action freqTab.freqTab / table={caslib="public" name="class_format_test"} 
  tabulate={
  	{vars={'ageGroup'}}
  }
  display={ names={"OneWayFreqs"} } 
  ;
run;
quit;

Re: apply user defined format UDF

acordes — Tue, 18 Feb 2020 09:33:26 GMT

I've finally achieved my goal:

There was one missing link, I had to promote the format by running the following lines

cas sugus savefmtlib fmtlibname=myFmtLib         /* 5 */
   table="myFmtLib.sashdat" caslib=casuser replace;

cas sugus addfmtlib fmtlibname=myFmtLib     /* 2 */
   table="myFmtLib.sashdat" caslib=casuser replacefmtlib;

cas sugus promotefmtlib fmtlibname=myFmtLib replace;

Re: apply user defined format UDF

acordes — Sat, 19 Sep 2020 18:56:09 GMT

@BrunoMueller could you please help me again? The same error occurs again despite I'd say me doing the same that brought me the solution last time. I've opened a question named UDF Visual Analytics.