Strategies for Reloading CAS Tables

4 Likes

Over time, SAS Cloud Analytic Services (CAS) servers accumulate in-memory data as users create SAS Visual Analytics reports, import data for programs, and load data to perform advanced analytics. When CAS servers stop, whether when individually restarted by an administrator, or when the SAS Viya platform is brought down, all of the data that was previously loaded by users and business processes is unloaded from memory.

One of the tasks all SAS Viya platform administrators must face is to develop a strategy for handling the reloading of CAS tables when CAS servers start to minimize disruption for their SAS Viya user community.

In this post, I will review a few options for SAS Viya administrators to employ as they formulate a reloading strategy. Chances are high that administrators will need to employ a combination of the following options to reload CAS tables for their users.

Strategy 1: Take advantage of just-in-time loading

Making sure that the CAS tables that back SAS Visual Analytics reports are readily available is often a primary concern for administrators when CAS servers start. Fortunately, SAS Visual Analytics will automatically load a CAS table for a report viewing user from any CAS library for which the user has LimitedPromote permissions.

Advantages

CAS tables are loaded only when needed.
Easy for administrators to implement -- just assign LimitedPromote permission for CAS libraries that commonly provide data for reports.
SAS Visual Analytics reports work automatically after every CAS start.

Potential concerns

The 'first viewer' penalty -- the first viewer of a report may have to wait for larger tables to load.
The automatic loading only happens for SAS Visual Analytics reports so CAS tables needed for other purposes will have to be managed separately.

Although a bit dated, you can read more about how this works in this SAS Communities article.

Strategy 2: Explicitly load tables

If the administrator is aware of certain tables that users rely on being available, they can modify the CAS startup configuration to explicitly load CAS tables after the CAS server starts but before users are allowed to connect to it.

To modify the startup script in the configuration instance, you will first need to add a reference to $deploy/sas-bases/overlays/sas-programming-environment/enable-admin-script-access.yaml in the transformers: section of your $deploy/kustomization.yaml.

Using SAS Environment Manager or the sas-viya command, administrators can configure the sas.cas.instance.config: startup property to include Lua code that explicitly loads CAS tables. This short section from the SAS documentation describes the startup property.

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

The administrator would need to modify the startup contents to include a Lua statement like this for each CAS table to be loaded:

---- Load HR summary table
s:table_loadTable{caslib="hrdl",
                  casOut={caslib="hrdl", replication=0.0},
                  path="hr_summary.csv",
                  promote=true
                 }

Advantages

CAS tables can be pre-loaded for any purpose whether that is for reports, analytics, etc.
There is no 'first viewer' penalty for CAS tables used for SAS Visual Analytics reports.
Frequently used large tables can be readily available for immediate use.

Potential concerns

Loading extremely large tables might impact CAS startup times. This depends on the CAS library backing data store and whether it supports parallel loading. See this article for examples.
Assumes the administrator knows which CAS tables need to be pre-loaded.
The list of CAS tables that need to be pre-loaded may change frequently.
There is a risk that the CAS table is loaded but not used.
Lua, which may be unfamiliar to some, is the only supported language for table loading instructions.

Strategy 3: Use CAS Table State Management jobs

CAS table state management jobs manage the importing, loading, and unloading of source files in CAS. SAS provides sample jobs for the Public CAS library that you can copy and modify to act upon other CAS libraries. CAS tables state management jobs can be quite flexible in terms of selecting a subset of files to load from a given CAS library.

Once defined, CAS tables state management jobs can be run manually in an ad-hoc manner or you can schedule them to recur. This can be beneficial if you have a CAS library with source files that are frequently updated or added. You can also chain jobs together for convenience purposes.

As far as using this as a re-loading strategy, the administrator can start the CAS server and rely on the next scheduled occurrence of the CAS table state management jobs to take care of loading tables. This works best if CAS servers are started far enough in advance of users accessing SAS Viya for the scheduled jobs to run.

If immediate loading is required, say for an ad-hoc CAS server restart, the administrator can manually execute any number of CAS table state management jobs to load CAS tables from the Jobs page of SAS Environment Manager.

For instructions on how to copy and modify the sample CAS table state management jobs, see this SAS Communities article or the SAS documentation for CAS table state management.

Advantages

Multiple source files from a single CAS library can be loaded.
Jobs can include filters that load a subset of source files.
State management jobs can be scheduled.
There is a sample job for unloading CAS tables that have not been used recently. This can be quite helpful in maintaining CAS servers that are long-lived.

Potential concerns

There may be a lag between the CAS server starting and the next execution of the scheduled jobs.
May require manual execution of the jobs by the administrator.

Strategy 4: Configure State-Transfer for CAS

This strategy is somewhat different as it only helps in cases where a CAS server needs to be restarted while keeping the rest of the SAS Viya platform running. It does nothing to help with reloading CAS tables from a cold start of the entire SAS Viya platform.

Enabling CAS for state transfer allows the sessions, tables, and state of a running CAS server to be preserved between a running CAS server and a new CAS server instance. Because CAS tables and sessions are transferred to the new instance, there is no need to reload any data.

Details and instructions for enabling CAS state transfer are available in SAS Viya - CAS Server Life Cycle Management on SAS Communities.

Advantages

Minimizes potential disruption for users.
CAS tables do not have to be reloaded.
Existing sessions tranfer to the new CAS instance.

Potential concerns

Does not help with reloading tables when the CAS server first starts.
Requires a persistent volume to use during the transfer of tables and sessions to the new instance.
May briefly elevate cluster resources as the transfer takes place.

Wrapping up

As mentioned earlier, I fully expect that administrators will need to employ a combination of these reloading strategies to properly implement a CAS reloading plan for their user communities. Administrators will have to weigh different user demands, service level agreement terms, and reasons for reloading CAS tables to determine the best strategy for their situation.

I hope that by describing common options and the advantages each has, administrators can make an informed decision as to the best plan of action for their deployments.

Find more articles from SAS Global Enablement and Learning here.

Zengahoe · ‎09-21-2023

Hi

I am trying to unload a table from memory in the caslib. Is it possible to use Prepare Data CASL option?

loadActionSet 'fedSql';
queryCode='drop table "'||_dp_inputCaslib||'"."TEST_SORT" {options force=true} ';

print queryCode;
ExecDirect / query=queryCode;

I try to run the prepare data query, the error is TEST_SORT cannot be found under the caslib. Please help. Thanks.

ScottMcCauley · ‎09-22-2023

Hi, that approach should work as long as the table you are dropping is loaded at the time. The message you mentioned suggests that the table was not loaded. A colleague of mine ran code based on your example and it ran as expected.

80   proc cas ;
81   loadActionSet 'fedSql' ;
82   _dp_inputCaslib='dm' ;
83   queryCode='drop table "' || _dp_inputCaslib || '"."ORDER_FACT" {options force=true}' ;
84   print queryCode ;
85   ExecDirect / query=queryCode ;
86   quit ;
NOTE: Active Session now MYSESSION.
NOTE: Added action set 'fedSql'.
drop table "dm"."ORDER_FACT" {options force=true}
NOTE: The FedSQL action was successful.
NOTE: PROCEDURE CAS used (Total process time):
      real time           0.12 seconds
      cpu time            0.01 seconds

You could also try an alternative approach such as:

80   proc cas ;
81   table.droptable / caslib="dm" name="order_fact" ;
82   quit ;
NOTE: Active Session now MYSESSION.
NOTE: Cloud Analytic Services dropped table order_fact from caslib DM.
NOTE: PROCEDURE CAS used (Total process time):
      real time           0.03 seconds
      cpu time            0.00 seconds

Hope that helps.