I’ve written about SAS Viya with SingleStore and showed how we can leverage data updates from SingleStore in near real-time in SAS Visual Analytics. In the coming posts, we will dive in a little bit deeper and understand some of the concepts behind the scenes. First let’s see how data flows from SingleStore to CAS.
In order to process data in SAS Viya, we usually have to load it into CAS first. This is true for most of SAS visual applications (like SAS Visual Analytics) which work on global tables. This is also true with code (if we put the “transient scope” aside for more simplicity). This means we create a copy of the data into CAS.
With SAS Viya with SingleStore, the SingleStore table is actually not loaded in CAS at load time (when we load it through the UI or with a loadTable CAS action). A few metadata are grabbed from SingleStore (table structure, columns, number of rows, etc.) and the table is then “declared” in CAS. No copy of data is done in CAS and no data movement occurs.
This is depicted in the following figure:
The above mechanism relies on the default value of the backingStore option which is INPLACEPREFERRED. If you set this option to CASDISKCACHE, you will replicate the behavior of traditional CASLIBs (duplicate the data in CAS at load).
In “traditional” CAS, CAS actions run on CAS data (the copy of the original table loaded in CAS) in the CAS server.
With SAS Viya with SingleStore, the CAS action processing will be distributed between SingleStore and CAS. The goal is to reduce data movement between SingleStore and CAS by preparing a subset of the data in SingleStore. As of today, SingleStore handles:
Technically, this is handled by the SAS Embedded Process deployed in the SingleStore cluster.
Then, with SAS Viya with SingleStore, the result set of the previous step is streamed dynamically from SingleStore to CAS for final computations.
Finally, the CAS action processing is much more than just a selection of fields and records and computation of additional columns. It is about simple or complex analytical processing that is performed in CAS on the data that has just been streamed in. The final results are sent to the calling client.
The entire flow description is also available in this video:
Thanks for reading.
Don’t miss the livestream kicking off May 7. It’s free. It’s easy. And it’s the best seat in the house.
Join us virtually with our complimentary SAS Innovate Digital Pass. Watch live or on-demand in multiple languages, with translations available to help you get the most out of every session.
The rapid growth of AI technologies is driving an AI skills gap and demand for AI talent. Ready to grow your AI literacy? SAS offers free ways to get started for beginners, business leaders, and analytics professionals of all skill levels. Your future self will thank you.