I’ve written about SAS Viya with SingleStore and showed how we can leverage data updates from SingleStore in near real-time in SAS Visual Analytics. In the coming posts, we will dive in a little bit deeper and understand some of the concepts behind the scenes. First let’s see how data flows from SingleStore to CAS.
In order to process data in SAS Viya, we usually have to load it into CAS first. This is true for most of SAS visual applications (like SAS Visual Analytics) which work on global tables. This is also true with code (if we put the “transient scope” aside for more simplicity). This means we create a copy of the data into CAS.
With SAS Viya with SingleStore, the SingleStore table is actually not loaded in CAS at load time (when we load it through the UI or with a loadTable CAS action). A few metadata are grabbed from SingleStore (table structure, columns, number of rows, etc.) and the table is then “declared” in CAS. No copy of data is done in CAS and no data movement occurs.
This is depicted in the following figure:
The above mechanism relies on the default value of the backingStore option which is INPLACEPREFERRED. If you set this option to CASDISKCACHE, you will replicate the behavior of traditional CASLIBs (duplicate the data in CAS at load).
In “traditional” CAS, CAS actions run on CAS data (the copy of the original table loaded in CAS) in the CAS server.
With SAS Viya with SingleStore, the CAS action processing will be distributed between SingleStore and CAS. The goal is to reduce data movement between SingleStore and CAS by preparing a subset of the data in SingleStore. As of today, SingleStore handles:
Technically, this is handled by the SAS Embedded Process deployed in the SingleStore cluster.
Then, with SAS Viya with SingleStore, the result set of the previous step is streamed dynamically from SingleStore to CAS for final computations.
Finally, the CAS action processing is much more than just a selection of fields and records and computation of additional columns. It is about simple or complex analytical processing that is performed in CAS on the data that has just been streamed in. The final results are sent to the calling client.
The entire flow description is also available in this video:
Thanks for reading.
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.