Just-In-Time Data Loading for Visual Analytics 8.1

[#1]

One of the biggest headaches for SAS administrators of SAS Visual Analytics 7.x environments was making sure that data needed for key reports got reloaded into memory on the LASR Server whenever servers restarted. Some admins crafted custom code to do this but many administrators took advantage of the autoload facility to simplify reloading data. Set up an autoload directory, schedule the loading job, drop crucial data to the directory, and voilà, the scheduled job would make sure the data were loaded into LASR. While this certainly works, there was sometimes a lag between server startup and the next autoload run which created the possibility of someone opening a report before the data was ready. With SAS Visual Analytics 8.1 and the magic of CAS libraries, making data available for reports happens automatically so administrators can rest easy.

To understand how this magic works in VA 8.1, it helps to know a bit about CAS libraries. If you are not familiar with CAS libraries, take a moment and watch Gerry Nelson's video, SAS Viya CAS Libraries (Caslibs) Simplified, which is a nice introduction to the topic. Go ahead, you won't regret it.

Ok, you've listened to Gerry and had your fill of cat videos, welcome back. You now know that CAS libraries are a logical reference to both the in-memory space as well as an associated persistent data store for a given set of data.



Therein lies the magic. For any given CAS library, the CAS Server knows where to get the source data to load into in-memory space if it is not already loaded. Brilliant, huh?

So how does this work? Let's say I work for a ruthless executive named, I don't know, Franz Lohman, who works around the clock and demands access to his key reports no matter the hour. As the SAS administrator, Mr. Lohman is going to have my head if his reports don't work. Fear not, I simply have to make sure the SAS Visual Analytics 8.1 environment is up and running and the CAS library definition will take care of the rest for me. Let's say that I happened to cycle my servers one Sunday night and bright and early Monday morning, Mr. Lohman accesses one of his reports requiring data that is not already loaded into memory. The CAS library definition for the data for his report enables VA and the CAS Server to locate the persisted copy of the data. The data are then automatically loaded from the persistent data store into memory and the report displays as he expected. This holds true whether the data are persisted in an operating system directory, HDFS, or a relational database. Pretty slick.

[Aside: Of course, if needed, I can still write a tiny bit of code that will pre-load data into memory to prevent the first viewer from being penalized but the point is that I don't have to. For example, I might consider the code option if I had extremely large files that I knew were going to be needed. I would probably pre-load something like that to keep the first viewer from having to wait too long.]

I'm sure SAS admins will appreciate this new feature VA 8.1 and no longer cringe when anyone mentions the word 'restart.'

[dmjones]

Thanks for writing this. Quick note, shouldn't RBDMS be RDBMS in the figure? I'm pretty sure you're referring to a Relational Database Management System, but I could be mistaken 🙂

[ScottMcCauley]

Yes, thanks for catching that typo.  It most definitely should be RDBMS.