We’re smarter together. Learn from this collection of community knowledge and add your expertise.

Autoload/unload/import of data via scheduled jobs in Visual Analytics 8.2

by SAS Employee GerryNelson on ‎03-13-2018 05:23 PM - edited on ‎03-14-2018 11:04 AM by Community Manager (2,193 Views)

Our old friend autoload from VA 7.x is back, well sort of. Autoload like capabilities have been added to VA 8.2 on Viya 3.3. In this blog I will look at what you can and cannot do in VA 8.2. The addition of this functionality is very useful, and it is a much simpler and cleaner implementation than autoload in VA 7.x. It is a nice addition to the existing just-in-time load functionality that Scott discussed in this post.


By now everyone should know what a CASLIB is. If you don't or need a refresher check out this video  SAS Viya CAS Libraries (Caslibs) Simplified


In VA 8.2 a user is able to create and schedule jobs. It is this functionality that can be used to automatically load/unload/refresh in-memory data. There are four types of jobs that can be created:


  • CAS table state management
  • SAS Data Explorer
  • SAS Data Studio
  • SAS Visual Analytics


In this blog we will look at CAS table state management jobs. Using thee job administrator can automate the import, load, refresh and unload of source files into memory in CAS. In SAS Environment Manager three sample jobs, that can be used to create additional jobs are provided. The sample jobs:


  • import source files to sashdat format in an existing CASLIB. Import supports CSV, SAS7BDAT and EXCEL
  • load caslib source files to in-memory tables. You can load new tables and refresh tables that already exist in-memory
  • unload tables from memory. Tables can be unloaded ether immediately, or based on when table was last accessed.


CAS table state management jobs are documented in the System Administration guide under Data > CAS Table state management.


The jobs allow an administrator to setup a schedule where tables are added to memory, existing tables are refreshed from their source, and tables are removed from memory when they are no longer needed. In addition, the import job can be used to read files in various formats and import them to sashdat format in a caslib's source.


The sample jobs are provided as a template from which to create new jobs. Users may not edit the sample jobs, but they can be copied to create a new job, and the new jobs settings can be edited.


The Jobs can then be scheduled in SAS Environment Manager, or via the scheduling command-line interface. Currently the scheduler in SAS Viya supports simple time based events. Looking at the process to load data there are two moving parts.

  • the job
  • the schedule

The diagram below is an example of a load job.





This short video below shows how to:

  • Copy an existing job
  • Edit the job arguments
  • Run and/or schedule the job
  • Check the status of the job and view the log



The trickiest step in the process is editing the job argument options.   The options are stored as a JSON-formatted string.




The documentation provides a tip which I have found very useful. Copy the json for options to a JSON editor, make the changes, and then copy the result back into options.  The copy paste method makes it less likely that you will make a mistake in the JSON which will cause your job to fail.


The addition of autoload-like capabilities in VA 8.2 is welcome. The functionality will allow administrators more control of when data in CAS is loaded/unloaded and refreshed. By combining the jobs more complex business processes can be implemented. For example, the import job could be scheduled in conjunction with the load job to read new data from an external process, load it to sashdat in a CASLIB's source and then subsequently lift it into memory refresh the data in-memory.


As I mentioned earlier these capabilities are documented in the in the Viya System Administration guide in the  CAS Table state management  section.      

by New Contributor bunceleung
on ‎03-19-2018 10:29 AM

good video

Your turn
Sign In!

Want to write an article? Sign in with your profile.

Looking for the Ask the Expert series? Find it in its new home: communities.sas.com/askexpert.