The way that the SAS Viya Cloud Analytic Services (CAS) Server manages data in memory is a complicated topic. There are many twists and turns to cover all of the nuances and possibilities. However, most of that behavior can be distilled down to a few simple rules which adequately describe the majority of situations. This post will provide a brief but illustrative guide to the inner workings of how data is managed in memory by CAS.
The Rules
Let's start with what you came here for:
-
№ 1. CAS analyzes data in SASHDAT format, regardless of the original source
Data loaded into CAS can come from a variety of sources and formats thanks to the SAS Data Connector technology. And of course, CAS can write that data back out into a myriad of other formats as well. Refer to the SAS documentation about Working with SAS Data Connectors for a lot more explanation.
However, the data that CAS has in-memory is exclusively in SASHDAT format, regardless of its original source or ultimate destination. The SASHDAT structure is optimized for distributed analytic processing.
-
№ 2. All in-memory data is memory mapped to a locally-accessible backing store
Memory mapping associates an address in RAM with a location in locally-accessible storage (hard disk, file system, etc.).
Most of the time, CAS will memory-map any data loaded into memory to the CAS_DISK_CACHE. But, CAS_DISK_CACHE is not the only backing store for in-memory data.
If the original source data is already unencrypted SASHDAT which is accessible via local storage, then CAS will memory-map to that location as the backing store instead - but just for the source data.
All changes to in-memory data are memory mapped exclusively to the CAS_DISK_CACHE.
-
№ 3. The OS, not CAS, moves data to/from the backing store
Here's an over-simplified explanation: CAS initially places the data either in RAM or on disk and then creates the memory map. CAS then references the place in memory for the data it wants. If the data is already there in RAM, then great, it's instantly available. Else the OS will "page in" the data from disk at the other end of the memory map. Similarly, if data in RAM is unused and the OS determines that it needs to free that space for other work, then the OS will automatically "page out" that data to the disk at the other end of the memory mapping. There are other situations not covered here, but the gist of it is that the OS is very efficient at managing data access with memory mappings.
This approach effectively provides a virtual memory scheme which increases the amount of data CAS can bring on-board and work with, reducing the need for manual administration of in-memory table availability.
-
№ 4. CAS_DISK_CACHE is where all replicated copies of data are stored
Besides acting as a virtual memory storage backend, CAS_DISK_CACHE also provides data protection for CAS workers. Additional copies of SASHDAT blocks can be stored in CAS_DISK_CACHE so that a complete SASHDAT table is still available if a CAS worker goes down (taking its allotment of data blocks with it).
Remember, these rules are just to describe a basic understanding of how CAS operates with respect to data in memory. There are many configuration and programming considerations available which can be used to address a wide spectrum of different scenarios that do not adhere to these rules. But I find that these rules provide a good starting point from which to explore those variations.
Implications
There are many implications which result from how CAS manages the data in memory. Let's take a quick look at a few of them.
CAS_DISK_CACHE is virtual memory... to a point
As a virtual memory scheme, the idea is that CAS_DISK_CACHE provides resilience on occasions when RAM is over-pressured with too much data. If needed, in-memory tables can be paged out to the CAS cache via the memory mappings. After all it's better for CAS to run slowly while accessing an infrequently used table than to fail for out-of-memory conditions. However, there are some CAS items for which there is no memory mapping - they must remain in memory for CAS to work. The point here is that CAS_DISK_CACHE is not sufficient for all virtual memory operations - so take care when relying on it.
It is important to realize that CAS_DISK_CACHE is a critical consideration in host machine setup. It must be accounted for in terms of size and performance with real-world hardware when properly sizing a SAS Viya solution. Read more about these considerations in my blog post, Provisioning CAS_DISK_CACHE For SAS Viya.
But virtual memory does not provide performance equivalence with RAM. It's much, much slower. To keep things snappy, it is more important to ensure the CAS hosts have sufficient RAM first and foremost. Prioritize your money on RAM first.
CAS_DISK_CACHE offers protection for CAS worker failure
When CAS employs COPIES=1 (or greater), then additional block replication of CAS data is performed with duplicates kept in CAS_DISK_CACHE. Hopefully, they're never needed, but if a CAS worker fails, and the users can continue working with uninterrupted access to their in-memory tables, then that's a great feature.
Lazy loading of SASHDAT
When CAS is pointed at a data source with data in SASHDAT format which is accessible on local disk, then it behaves differently by memory mapping to the source location directly (and so it doesn't use CAS_DISK_CACHE to back those blocks). Building the memory maps is a minor and quick action. So when the loadTable directive is executed, it seems as if CAS loads that SASHDAT table extremely fast - way faster than physically possible. That's because it just builds those memory maps at that time. It's when the data is first accessed by CAS for analytics work that the OS realizes that the data isn't yet in RAM, so it pages in the data from the source disk (meaning the first analytics job must wait longer than any subsequent ones). For really large SASHDAT tables, consider running a preliminary job that will force the OS to page in the data before real end users try to hit it.
Default COPIES=1, except when it's 0
By default for most external data sources (i.e. not locally-accessible SASHDAT), CAS uses COPIES=1 to place redundant blocks of data in the CAS_DISK_CACHE to guard against failure of a single CAS worker. But there's one exception to that rule: loading SASHDAT data from remote HDFS. CAS cannot memory map to remote HDFS blocks of SASHDAT, so if CAS loses a worker in this scenario, then the in-memory table will be corrupted, requiring a reload. The assumption here, however, is that remote HDFS will be fast enough to reload the data easily enough that it's worth the tradeoff and avoid using CAS_DISK_CACHE. It is possible to manually specify COPIES=1 or any other preferred value.
DNFS and CAS_DISK_CACHE are not the same
Both DNFS and CAS_DISK_CACHE can be considered as locally-accessible storage which can work with CAS' use of memory maps to associate data in RAM with locations on storage devices. But their actual implementations will likely be very different.
DNFS is a SAS-term simply meaning shared file system. There are a wide ranges of offerings and options which work perfectly fine for DNFS (assuming their performance meets expectations). For example, these offerings can range from plain old NFS to high-performance clustered file systems. One such CFS is IBM SpectrumScale (a.k.a. GPFS) which is often a go-to technology used for many high-performance SAS solutions.
CAS_DISK_CACHE, on the other hand, will definitely not be any kind of shared file system. It must use local disk(s) which are ideally provisioned exclusively for the CAS host. CAS_DISK_CACHE supports specifying multiple directory paths and so high-performance and high-throughput are possible here, too. But the techniques that CAS uses to work with files in its cache isn't compatible with many shared file system technologies. Never use an NFS-based solution for CAS_DISK_CACHE. And GPFS simply won't work.
Point is, plan and provision storage space for DNFS and CAS_DISK_CACHE separate from each other. They are not interchangeable. This concept also carries over to Viya deployments where CAS is symmetrically colocated alongside Apache HDFS. The challenge there can be ensuring enough disk space is available for CAS_DISK_CACHE (i.e. that it's not all given to HDFS).
For a lot more detail about performance of storage performance, see Tony Brown's paper from SAS Global Forum, Engineering CAS Performance Hardware Network, and Storage Considerations for CAS Servers.
Monitoring
There are several tools which can provide insights into how CAS is managing its data in memory. For a normal SAS user, this is likely more detail than is necessary. However for Viya administration and other IT personnel, there are details which might come in handy in certain situations.
To get started, take a look at the SAS documentation for Monitoring: How to (CAS Server Monitor).
