SAS Viya CAS Format library persistence
User-defined format in CAS and Viya was a gold mine for early GEL blogs. This was because the way CAS accesses formats is very different than BASE SAS. To summarize, in Viya, if a CAS table has columns that reference user-defined formats, the formats must be made available to the CAS server. CAS cannot directly read SAS catalogs that store user-defined formats. SAS format catalogs must be converted to a CAS format library and stored in a sashdat file in a path-based CASLIB. In addition, to be made available to all users, the CAS format library must be promoted in a global CASLIB. In this article, I will look at how life has become easier for the Viya Administrator in relation to user-defined formats and CAS.
In a previous article, I described some of the interfaces that an administrator can use to make user-defined formats available. The interfaces include:
- SAS Environment Manager
- The CAS plugin of the admin/viya command-line interface
- SAS Code using statements, proc CAS, cCAS actions and/or proc casutil.
In previous releases, the methods above would make user-defined formats available to the CAS server, however, access to the formats would not persist across CAS server restarts. The only way to persist the availability of user-defined formats across server restarts was to add lua code to the CAS server start-up script files (casstartup.lua and casstartup_usermods.lua) and in the code:
- Load the format library into memory in the global FORMATS caslib
- Add the format library to the CAS server format search path.
What has changed?
In Viya 4 you no longer need to update the CAS startup files. Now, regardless of the interface that you use, when you perform the two steps above, access to format libraries is persisted across CAS server restarts. (Side note: changing server startup files is very different in Viya 4 as described in this article).
Let's see how it works.
The CAS permstore is a directory that contains the metadata about CAS including Caslib definitions, permissions, etc. In Viya 4, the permstore is located on a persistent volume and is accessible inside the CAS controller pod at the location /cas/permstore. For more details on storage and CAS check out this article. Persistence of access to formats now works automatically with the help of the CAS permstore. Now when you add a format library in Environment Manager or with any of the other interfaces two files are written to the permstore. These files are persisted in the permstore and used every time CAS starts to configure access to user-defined formats. To see this in action, first, let's use a kubectl command to get the name of our CAS controller pod.
kubectl get pod -l casoperator.sas.com/node-type=controller --output=jsonpath='{.items..metadata.name} {"\n"}'
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
Next, we will exec into the CAS controller pod and check the contents of the permstore.
kubectl exec -it sas-cas-server-default-controller -c cas -- ls -li /cas/permstore/primaryctrl
All good so far, now let's use the CAS plug-in to the sas-viya command-line interface to:
- create format library
- import formats from a SAS catalog
- update the format search order
The command below performs all these tasks, AND now persists the changes in the CAS permstore. If you would like to see how to do this in Environment Manager check out my previous article.
sas-viya cas format-libraries create --server cas-shared-default --format-library slformats --search-order append --source-path /gelcontent/gelcorp/sales/formats/formats.sas7bcat --su
Let's repeat the command we used to view the contents of the permstore. In the permstore you can see two new files setfmtsearch_startup.lua and addfmtlibs_startup.lua.
kubectl exec -it sas-cas-server-default-controller -c cas -- ls -li /cas/permstore/primaryctrl
If we view the contents of the files we will see that they contain the code to make the format library available and add it to the search path. This is the same code that, in previous releases, you would have had to add to the CAS startup files as an additional step. addfmtlibs_startup.lua
kubectl exec -it sas-cas-server-default-controller -c cas -- cat /cas/permstore/primaryctrl/addfmtlibs_startup.lua
setfmtsearch_startup.lua
kubectl exec -it sas-cas-server-default-controller -c cas -- cat /cas/permstore/primaryctrl/setfmtsearch_startup.lua
These two files are now generated/updated automatically when a format library is created and/or added to the search path. They are persisted in the permstore and used to restore access to the formats when the CAS server restarts. No need any longer to update the CAS startup files. This is a really nice improvement in Viya ?????
Summary
The documentation sums the change up perfectly:
"In SAS Viya, global-scope user-defined format libraries that are created, saved, and promoted by a SAS administrator, are automatically preserved across CAS server restarts, transfers, and migrations. SAS Viya 3.5 and earlier releases required the use of start-up script files (casstartup.lua and casstartup_usermods.lua) to add and promote global-scope user-defined format libraries across CAS server restarts."
Find more articles from SAS Global Enablement and Learning here.
