The GEL team recently moved our internal reporting environment from Visual Analytics on 9.4 to Visual Analytics on Viya. We used the 9.4 to Viya promotion tools documented here to move the reports and explorations. As part of the process we wanted to keep the two environments running in parallel. Running the environments in parallel is very helpful in validating the content and appearance of the reports after they are imported to Viya. One of the really helpful things in keeping the two environments running in parallel was the ability to keep the data in LASR and refresh the CAS data from the 9.4 LASR server.   SAS Viya promotion tools support export and import of LASR servers and libraries from SAS 9.4 to Viya. In the article I will show how this is done by exporting a LASR Server and library from SAS 9.4, importing them to Viya.   Exporting LASR Servers and Libraries Viewing the LASR server in SAS 9.4 Visual Analytics administration show that it has two in-memory tables.   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   In SAS 9.4 use the import/export wizard in SAS Management Console to export the LASR servers to Viya. On the Folders tab in SAS Management Console, navigate to SAS Folders > System > Servers, right-click and select Export SAS Package and give the package a name.   Repeat the process to export the HR Lasr Library from SAS 9.4. to a different package file.   Importing LASR Servers and Libraries to Viya The LASR server package is not actually imported to Viya, it is used in the import process to create properties within a Viya Mapping File. The properties are then used to provide default values for a new Caslib which will connect to the 9.4  LASR server. To create the initial mapping file use the transfer command-line interface.   sas-admin transfer generate-content-mapping --mapping c:\temp --server-package "C:\Users\94packages\lasr_servers.spk"   By default the mapping file is called ContentMapping.json.     The mapping file contains substitution properties that will become properties of a new LASR based Caslib in Viya. The file is then used in the next step when the LASR library is imported.   The LASR Library package can be imported in SAS Environment Manager or using the transfer CLI. In SAS Environment Manager, in the content area select Import and select the LASR library package. In the import wizard Mapping tab openthe mapping file that was created from the LASR server.   In the Additional Mappings tab map the LASR server to the CAS server.     Under Substitution Properties keep the new values and old values the same.     Complete the import process to create your LASR based CASLIB. The mapping file will be supplemented by the process and you can save a new mapping file at the end of the process.     Before you use it the LASR Caslib will require a Viya authentication domain. An authentication domain makes a set of credentials available to a set of users, in this case the credentials are those needed to access the SAS 9.4 LASR Server. The authentication domain was not created by the import process, it can be created in the SAS Environment Manager Security area.   With the authentication domain defined, if you browse to the LASR based Caslib in SAS Environment manager you should see the LASR tables in the source of the LASR Caslib.   Loading Data from LASR to CAS The data can be loaded into memory in CAS from the LASR server using the new caslib.   To transfer the data to CAS, create a new path-based Caslib an a cross-load from the LASR based caslib to the new path-based caslib. This will take the data from LASR, load it into the memory inf the new  path-based caslb and then persist it as a sashdat file in the source directory . In the future we can load directly from the source of the path-based CASLIB or refresh the data from the LASR server.   The program below uses PROC CAS to: Load the LASR data from the LASR Caslib into memory in the path-based Caslib Persist the data as a sashdat file in the source path of the path-based Caslib cas dataimportsess; proc cas; /* drop the current table if it exists */ table.droptable / caslib=”HR Caslib” name=”LONGCOMMUTE” quiet=”TRUE”; table.droptable / caslib=”HR Caslib” name=”HRDATA” quiet=”TRUE”; /* Load the table from the LASR caslib to the memory space of the target caslib */ table.loadtable / caslib=”HR – LASR Library” path=”LONGCOMMUTE” casout={caslib=”HR Caslib”,name=”LONGCOMMUTE”,promote=TRUE}; table.loadtable / caslib=”HR – LASR Library” path=”HRDATA” casout={caslib=”HR Caslib”,name=”HRDATA”,promote=TRUE}; /* save the table as a caslib in the source of he target path caslib */ table.save / caslib=”HR Caslib” name=”LONGCOMMUTE” table={caslib=”HR Caslib”,name=”LONGCOMMUTE”} permission=”GROUPWRITEPUBLICREAD” replace=True; table.save / caslib=”HR Caslib” name=”HRDATA” table={caslib=”Shared Data”,name=”HRDATA”} permission=”GROUPWRITEPUBLICREAD” replace=True; quit; cas dataimportsess terminate;   It can be very convenient when transitioning from 9.4 and LASR to Viya and CAS to keep your data in LASR and access it in CAS. As you have seen, it is also quite easy when the time comes to move the data completely into the CAS server. I hope you find the useful. ... View more

Promotion is the process of moving resources from between environments. In SAS Viya promotion of content within folders like reports, data plans, or models is accomplished using the sas-admin transfer command-line interface, or the import and export wizards in the SAS Environment content area. There are some resources in SAS Viya that don't reside in folders which it may also be desirable to promote between environments.   In this post I will look at exporting and importing content that does not live in folders using the sas-admin transfer command-line interface (CLI) and SAS Environment Manager. In SAS Viya, a uniform resource identifier (URI) is used to identify unique resources. The transfer plug-in of the sas-admin cli has the flexibility to be able to export based on the URI of a resource in SAS Viya. In this post we will look at promoting custom groups and general authorization system rules.   In SAS Viya, most identities are read from an LDAP server. The exception is custom groups which are groups that exist in SAS Viya but not in the LDAP server. There are some default custom groups that come with the software, and administrators can also create their own custom groups. When would you want to promote custom groups? You may want to export and import custom groups when you have a dev/test/prod or multi-tenant scenario and you want to replicate your custom groups in a separate environment.   In the scenario used in that article, I have implemented a report developers group and I want to promote my group to production. The report developers group is a group created to support separating users into report viewers, who can only view reports in SAS Viya, and report developers, who can create reports in Visual Analytics. This setup is supported by a rule in the SAS Viya authorization system. The process of doing this is illustrated in that article.   Let's look at how we would promote this setup one environment to another. The screen shot below shows the report developers custom groups which contains three member custom groups.   Click any image to see a larger version.   In addition there is a rule that grants read access to the Visual Analytics application only to members of the Report Developers group. Since SAS Viya authorization has an implicit deny, this means that anyone who is not a member of the Report Developers group cannot build reports in Visual Analytics.   To export the groups and the rule we will need the URI of the resources we are exporting. The first step for custom groups is to get there "id" so that we can construct the URI. We could get this from SAS Environment Manager, or use the sas-admin identities command-line interfaces (CLI).   /opt/sas/viya/home/bin/sas-admin --output text identities list-groups     Similarly, we need the id of the general authorization system rule. In SAS Environment Manager we can get the id, by right-clicking on the rule and selecting properties. The id of the rule is the last item.     Following that we need to build a json file that will define what will be exported. The format of the json file can be found in the SAS Viya administration guide.   Here is the file we will use to export the rule and the custom groups. The items array contains the list of URI's of any resource supported by the transfer cli, e,g report, data plan, folder, rule. In the previous step we returned the id, which is the last element in the URI. The first part of the URI is the rest endpoint of the resource. If you don't know the REST endpoint you can find documentation on developer.sas.com.   { "version": 1, "name": "My custom groups", "description": "Export Viya Custom groups and Rule", "items": [ "/identities/groups/Finance Content Developers", "/identities/groups/HR Content Developers", "/identities/groups/Sales Content Developers", "/identities/groups/ReportDevelopers" , "/authorization/rules/00b08a2f-847f-4c2a-98b5-e1711860d02d" ] }   To perform the export, as a SAS Administrator, use the following command. You can pass the request as a string, or when you include the information in a file, the first character following the request option must be the at sign (@). Therefore, if a pathname is used, it must start with the at sign (@).   /opt/sas/viya/home/bin/sas-admin transfer export --request @/tmp/export_groups.json     The export package is created in the SAS Viya environment, to move it to your target environment download it, passing the id of the package to the download command.   /opt/sas/viya/home/bin/sas-admin transfer download --id 2916d2cb-14bd-4418-9aff-2d90feb4397e --file /tmp/package_groups.json   This creates a SAS Viya transfer package json file that can be imported to the another environment. You can open the JSON file and view the content. There should a JSON representation of each object in the package.     Copy the json file to the target environment. In the target environment you have two choices for import. You can import with the command-line interface or with SAS Environment Manager. Let's look at SAS Environment Manager first.   As a SAS Administrator in the Content Area select import and select the package. Walk through the wizard (there are no mappings) to import the resources. View the status of the import.     During import any custom group that did not exist in the target environment is created. If a custom group did exist it is not imported. Group membership is preserved as long as the the members of the group exist in the target environment as either LDAP identities or Custom groups.   The import of the rule behaves differently: If a rule with the same id exists the contents of the rule are updated, if no rule exists the rule is added.   The same import process can also be accomplished with the transfer command-line interface. Using the CLI there are a few more steps. However, you can script a work around for the inability to replace existing groups by deleting any custom groups that exist prior to the import. For example:   /opt/sas/viya/home/bin/sas-admin identities delete-group --id "ReportDevelopers"   To import with the CLI, upload the package to the target SAS Viya environment.   /opt/sas/viya/home/bin/sas-admin transfer upload --file /tmp/package_groups.json     After the upload do an import, passing the id of the uploaded package. The message should indicate that the 5 resources have been successfully imported.   /opt/sas/viya/home/bin/sas-admin transfer import --id <id>   For more information on Promotion and the transfer CLI, take a look at: SAS Viya Administration Guide: Transfer CLI SAS Viya Administration Guide: Promotion Thanks for reading. I hope you will find this information valuable. ... View more

In SAS Viya 3.4, there are two versions of SAS Studio. SAS Studio 5.1 is the new microservices-based version with a new interface. It integrates with other SAS Viya components (SAS Drive, Launcher & Compute) and is available in full deployments. Edoardo Riva wrote an article covering some of the architectural differences between the two SAS Studios. The different architecture of SAS Studio 5.1 means that it also has some functional differences from the non-micro service based versions of SAS Studio. One of the major differences is that SAS Studio 5.1 cannot open and save programs from the the OS file system.   This is a big change for existing SAS Studio users who are used to saving and opening their SAS code from directories on the file system. SAS Studio 5.1 instead allows users to load and save SAS programs to and from SAS Viya folders. The programs themselves are stored in the SAS Viya infrastructure data server using the file service.   For users who have a lot of SAS programs stored on the file system, this will initially be inconvenient. Fortunately, there is a fairly easy way to copy programs (or any other files for that matter) from the OS file system to a SAS Viya folder and vice-versa.   The filename statement includes the new file service (FILESRVC) access method that enables SAS programs to store and retrieve content using the SAS Viya Files service. The SAS Viya files service provides persistence of files, such as SAS programs, logs comment attachments, report images, etc.   There are two methods available to access files. You can access files directly using the file uniform resource identifier (URI). The URI is how SAS Viya uniquely identifies resources. You can also access files using the collection option, which is very useful when your files are attached to folders. The collection option means FILSERVC access method can both store files and associate files to folders.   So how do we use this filename to copy a file from a directory on the filesystem to a SAS Viya folder? The SAS program below uses: filename statement to access files in a LINUX directory a second filename statement with the files service access method to access a SAS Viya folder the fcopy function to copy the files between the two locations. /* filename to a sas program on the file system */ filename src "~/check_cas_auth.sas"; /* filename using FILESRVC to the destination a sas program and folder */ filename dest filesrvc folderpath="/Public" filename="myprogram.sas" debug=http; /* copy the file: output return code and any message */ data _null_; rc=fcopy("src","dest"); msg=sysmsg(); put rc=; put msg=; run;   Simple as that! When you run the program, the file is copied from the home directory of the user on Linux to the Public folder in SAS Viya. SAS Studio 5 users can now open, edit, and run the program from its location with a SAS Viya folder.   The debug=http option on the FILESRVC filename statement is useful when you have problems, and to see what is happening behind the scenes. The extract from the log below shows the end result: a file created in the file service and referenced within the folder requested.      Click any image to see a larger version.   The SAS program is now in the file service and can be surface in SAS Studio 5 or SAS Drive.     By reversing the source and destination it is also possible to go the other way and copy a file from SAS Viya to the operating system.   The logical extension of this approach is to copy a complete directory of SAS programs from the operating system to a folder in SAS Viya. The attachment below contains a SAS macro that will copy all the SAS programs in a OS directory to a folder in SAS Viya. It basically loops the directory content and submits the sample code shown above.   This all works well from SAS Studio 5.1 on SAS Viya, which is completely integrated with the SAS Viya micro-services and authentication. There are a couple of additional steps required to access the file service from SAS 9.4 M6 or earlier versions of SAS Studio.   Firstly, the servicesbaseurl SAS option must be set. The servicesbaseurl option tells the SAS session the root URL for calls to SAS Viya services. Servicesbaseurl is an invocation option, meaning it can only be set when SAS is starting. To set the servicesbaseurl, modify the SAS configuration file of your SAS session and add the following line, referring to your SAS Viya server:   -SERVICESBASEURL http://myserver.mysys.com   In addition, when using anything other than SAS Studio 5.1, you must have authenticated to SAS Viya and provided your session access to the token by setting the SAS_VIYA_TOKEN environment variable. The SAS_VIYA_TOKEN environment variable can be set at invocation time, or in a options statement within your SAS code. For example:   option set SAS_VIYA_TOKEN=<your token>   There are a number of approaches you can use to get the authentication token in your SAS session. The attachment includes two macros: %authtoviya authenticates to SAS Viya using proc http and sets the SAS_VIYA_TOKEN environment variable. %getaccesstoken assumes you have authenticated using the sas-admin CLI and reads the access token to set the SAS_VIYA_TOKEN environment variable. The transition to using SAS Viya folders instead of operating system directories can be made easier using the file service access method of the filename statement. In addition, the access method can be used to read and write may other file types to and from a SAS Viya environment.   I hope you find this useful. Here are some additional resources: SAS Studio documentation One, Two, Three SAS Studio! Enter SAS Studio 5. FILENAME Statement, FILESRVC Access Method servicesbaseurl SAS_VIYA_TOKEN environment variable File service Folders service ... View more

Your identity to a computer system is one of the key factors that determine what you can and cannot do, or what you are "authorized" to do. In this article we will look at identity management in SAS Viya visual deployments (programing only deployments are different).   My colleague Stuart Rogers has covered in detail the authentication aspects of SAS Viya, how you get in. In this article we will look at how your identity determines what you can do once you have authenticated.   SAS Viya Visual deployments establish a user's identity by reading user and group information from an identity provider. For SAS Viya 3.2 LDAP based identity providers are supported, including Microsoft Active Directory, OpenLDAP or Apache Directory Server. Access to the identity provider can be configured in two ways: During deployment by entering the LDAP configuration information in SiteDefault.yml Post-deployment interactively with SAS Environment Manager Learn more about how to configure the connection to the identity provider.   There are no users stored in SAS Viya, but a key feature is the ability within SAS Viya to create custom groups. Custom groups can be created that exist in SAS Viya and not in the authentication provider. This provides great flexibility in creating groups to manage access to resources within SAS Viya.   There are 4 pre-defined custom groups in SAS Visual Analytics on SAS Viya.   SAS Viya Pre-defined Custom Groups     In addition to the four pre-defined groups authenticated users is an implicit group to which all users who can authenticate to the system belong.   To slightly complicate things the CAS server also has group-like structures called roles. There is a CAS administrator (or super user ) role and a Data administrator role. CAS Roles determine the level of administrative access for CASLIB, table, column and action administration and authorization Custom groups determine the level of administrative access in SAS Environment Manager for general administration and authorization CAS Roles and SAS Viya groups are related. By default in SAS Viya the SAS Administrator pre-defined group is included in the CAS Administrator role. Resulting in all SAS Administrators also being CAS Administrators.   CAS Roles and Role Membership in CAS Server Monitor     The table below shows the initial setup of the 4 SAS Viya pre-defined custom groups and what functionality they can access.     The SAS Administrators group is a special group called an assumable group. This means that a user can opt into that group to gain its elevated privileges.  In SAS Environment Manager if you opt into the SAS administrators group at logon you may also assume the CAS administrator(or super user) role to administer the CAS server. My colleague Scott McCauley covered assumable groups in a previous article.   What a user or group can do (and see) is controlled by rules.  A rule is a composite of authorization elements including: Target: a resource for example a folder or report Principal: user or group Permissions: type of access for example read or write Setting: indication of whether access is provided, for example grant or prohibit. SAS provides an initial set of rules to control your users’ access to resources.  These rules determine what the pre-defined custom groups can do and see. The screenshot below shows the default rules for the SAS Visual Analytics and SAS Visual Data Builder applications.   It shows that Authenticated Users (principal) can access SAS Visual Analytics (target) because Read (permission) is Granted (setting).   The DataBuilders pre-defined custom group(prinicpal) can access SAS Visual Data Builder(target) because Read (permission) is Granted (setting).     The general authorization system in SAS Viya implements a default deny to resources. A user cannot access a resource unless permissions is granted. For that reason, with the rules above applied, only members of the DataBuilders group can access the SAS Visual Data Builder application.   I hope this brief introduction to users and groups in SAS Viya will prove useful. In a follow up article I will go into more detail on rules, and show how you can create new groups and rules to change a user's access to functionality. ... View more

If a CAS table has columns which reference user-defined formats, the formats must be made available to the CAS server. CAS cannot directly read SAS catalogs that store user-defined formats. They must be converted to a CAS format library and stored in a sashdat file in a path based CASLIB. In addition, to be made available to all users, the CAS format library must be promoted in a global CASLIB.   In SAS Viya 3.4 we have some new functionality to make it easier to manage the availability of user-defined formats in both Environment Manager and the command-line interfaces.   Firstly, let's look at the new out-of-the-box configuration for user-defined formats. The standard install sets up: One global path based CAS library named Formats located at /opt/sas/viya/config/data/cas/default/formats/ One CAS format library, named sassuppliedformats (sassuppliedformats.sashdat) The CASLIB and CAS Format library are made available by being defined in the casstartup.lua file. This file runs when the CAS server starts. It is not recommended to make any changes to casstartup.lua (use casstartup_usermods.lua) or to load your own formats to sassuppliedformats. These files may be changed during a software update or  upgrade.   SAS Environment Manager In SAS Viya 3.4 in SAS Environment Manager you must be a SAS Administrator to manage user defined formats. An administrator can manage formats in existing format libraries including: Create new formats. Copy and delete formats. Import formats from an itemstore file accessible from the CAS controller SAS catalog accessible from the machine that runs the compute service OR a table sashdat file accessible from the CAS Controller in an SMP environment and the CAS Controller and workers in an MPP environment The ability to import formats from a sashdat table and directly from a SAS catalog is new in SAS Viya 3.4. The addition of the ability to import formats from a SAS catalog is a great new feature.   In addition, if a user is also a CAS administrator, and if they assume the super user role in SAS Environment Manager they can manage CAS format libraries. In SAS Viya 3.4 SAS Environment Manager a CAS administrator can: Create a format library (optionally populate it from an itemstore, SAS Catalog or table sashdat file) Delete a format library Update the format search path. The short video below will demo how to: Create a new empty CAS format library in SAS Environment Manager Import formats to the new CAS format library       The approach in the video is useful if you want to view the formats your are importing, and compare them to existing formats. In addition you don't need to be a CAS administrator to do the second step, importing the formats. This means that you can split the functionality so that only a CAS administrator can manage format libraries but other administrators can manage formats within the libraries.   An alternative approach, creates the CAS format library and imports the formats from a supported file all in one step.     This method is more efficient when you have a lot of formats in your catalogs, but you don't get a chance to see the formats, and you do have to be a CAS administrator.   The ability to make these changes in SAS Environment manager is a great improvement. However, you should be aware that the only way to persist the availability of user-defined formats across server restarts is to modify the cassstartup_usermods.lua file to: Load the format library into memory in the global FORMATS caslib Add the format library to the CAS server format search path. The image below shows the code to load the CAS format library into memory and add it to the search order in casstartup_usermods.lua. This file is runs every time the CAS server starts.     Making these changes will ensure that the CAS Format library is loaded into memory, and added to the formats search order every time the CAS Server starts. The format search order is used during session startup to automatically established format libraries to search when a table a format is needed.   In SAS Viya 3.4 the CAS plug-in to the sas-admin command-line interface has been enhanced to allow the administrator to create format libraries, update the format search order and import formats. The command-below perform all these tasks, however, the CLI also does not persist the changes, the only way to persist the changes is to include them in CAS server startup processing.   ./sas-admin cas format-libraries create --server cas-shared-default --format-library hrformats --search-order prepend --source-path /gelcontent/gelcorp/hr/formats/formats.sas7bcat --su   These changes give the SAS Viya administrator more choices for how they will make user-defined formats available to CAS in SAS Viya 3.4.    For more information check out the User Defined formats section in the Viya Administration Guide h ... View more

In a previous article, New functionality for transitioning from Visual Analytics on 9.4 to Viya, I discussed promotion of content from SAS 9.4 to Viya. The Viya promotion tools, by default, include authorization settings in the package when you export folders and content. What if you want to export the authorization separate from the content, and then selectively apply that authorization?  The good news is the SAS 9.4 and Viya provide us with the tools to take an alternate approach.   In this post I will look at how this can be accomplished. Recall that there are two authorization systems in SAS Viya. The General Authorization system applies to folders, content and functionality and the CAS authorization applies to data in CAS. Since the new authorization systems are very different from metadata authorization in SAS 9.4, the promotion tools attempt to translate the 9.4 settings to Viya settings.   General Authorization (Folders and Content) Promotion will attempt to convert SAS 9.4 authorization to rules in the General authorization system. During the process: Access Control Templates are not promoted Explicit Access Control Entries are converted to SAS Viya Rules Access Control Entries with denials are discarded CAS Authorization (CASLIBs and Data) Promotion will attempt to convert SAS 9.4 authorization to access controls in the CAS authorization system. During the process: Access Control Templates are not promoted Access Control Entries are not promoted unless they are applied directly to a library or table Access Control Entries are converted to CAS access controls Row-level permissions are preserved If an object exists in the target environment no authorization settings are imported. Why would we want to selectively promote authorization? Well there are a number of reasons. Instead of a big bang maybe we want to take a more incremental approach to the transition. Perhaps our 9.4 security model relied heavily on ACT's which are currently not supported. Maybe we want to change the way our content is structure and reapply security after that process is completed.   Fortunately we do have the tools available selectively promote authorization from 9.4 to Viya and at the same time include ACT's. There are three steps: Read SAS 9.4 authorization settings: this can be accomplished using the SAS 9.4 security reporting macros. Translate SAS 9 authorization to Viya: this can be accomplished using SAS data step and procedures and the mapping of SAS 9 permissions to Viya permissions in the Viya administration guide. Apply the Viya authorization settings: this can be accomplished using the sas-admin command-line interface. General Authorization The authorization plug-in of the sas adminstration cli's support the creation and update of rules in bulk. The create-rules command accepts a JSON file as input. For example the command below will add rules based on the content of the json file:   sas-admin authorization create-rules --file addrules.json   In the JSON file you define the details of the rules including the target folder, the principal the type of rule (grant or deny) and the permissions. So if we can create a json file from the 9.4 metadata we can export rules from SAS 9.4 and import to Viya.   This SAS macro will read 9.4 effective authorization for a folder and a set of identities, translate it, and output a json file of Viya rule definitions. It accepts as parameters: Metadata Folder Name of a json file to output A folder id for a Viya folder (optional) A list of identities A matching list of identity types Using the parameters the macro: Uses the SAS 9.4 security reporting macros to read the authorization settings for the folder and write them to a SAS table. Post processes the SAS table to map a SAS 9.4 permission to a Viya permissions (using the mappings provided in the Viya administration guide) Reads the resultant SAS table and uses proc json (introduced in 9.4 M4 ) to create a json file with the Viya authorization rule definition. For example, say we want to set permissions for a folder in Viya  /gelcorp/sales with the same settings we had for a related folder /gelcorp/salescontent.   Firstly, we can optionally determine the folder id of the target folder in Viya.   sas-admin folders show -path /gelcorp/salescontent     Next, in SAS 9.4, execute the macro to export the authorization settings from SAS 9.4. The macro will extract the  settings for one folder and can retrieve effective permissions for multiple groups on the folder. If you do not know the uri of the target folder you can leave it blank and edit the json at a later time.   %export_folder_auth(metadata_folder=/gelcorp/salescontent, output_file=c:\temp\gelcorpsales.json, folder_id=%nrstr(eacf2e97-caf5-43d0-9b66-f1e0b5f7de5c), identity=%nrstr(Sales,Sales Content Developers, Sales Data Managers), identitytype=%nrstr(IdentityGroup,IdentityGroup,IdentityGroup),);   The macro is included as an attachment.   The resulting JSON is shown below. If anything is incorrect in the JSON you can edit it now, the macro assumes initially that the groups exist with the same id in Viya. However, you can edit the json before you apply it to fix an issues such as different identity information or folder id.     To apply the permissions, use the authorization plug-in of the sas-admin CLI.   sas-admin authorization create-rules --file c:\temp\gelcorpsales.json   The result is new rules are created in Viya. You can view the rules in Rules Manager  by filtering on the objectUri and/or containerUri  and view the impact of the rules on the folders permissions in the content area.     The macro is deliberately written to do one folder at a time. The differences between 9.4 and Viya authorization make a bulk approach without any review dangerous. Doing the export one folder at a time means the new rules can be thoroughly reviewed prior to importing to Viya.   CAS Authorization CAS Authorization has a similar bulk load facility. The command below will replace the existing access controls on a caslib with the ones defined in the json file.   sas-admin cas tables replace-controls --server cas-shared-default --caslib mycaslib --table tableA --source-file accesscontrol.json   This is not quite what we need because we want to add controls to the CASLIB and keep the existing ones. To add controls you use the add-control command.   Our macro this time works in the same way to read and translate the authorization settings from 9.4 to Viya. However, instead of generating a JSON file. It generates a series of CAS  add-control cli calls in a script that will apply the permissions.   The macro this time accepts as parameters the: SAS Library metadata name Output file for the script Target caslib name A list of identities A matching list of identity types %export_library_auth(library_name=Financial Data, output_file=c:\temp\mylibrules.bat, caslibname=Financial Data, identity=%nrstr(Finance Content Developers, Finance Data Managers,Finance), identitytype=%nrstr(IdentityGroup,IdentityGroup,IdentityGroup));   The macro is included as an attachment.   The output is a script file (Windows or LINUX) containing add-control requests for the groups and caslibs included in the macro call. The script can be executed by a CAS administrator to set caslib permissions.     The result can be viewed in the data area of SAS Environment Manager.     Simple as that! I hope the approach described in this article will be useful to those wishing to move to Viya and translate their 9.4 security model. Here are some useful resources: Security Reporting Macros in 9.4 JSON Procedure Viya Administration Guide: Command line-interfaces Viya Administration Guide: Promotion Viya Administration Guide: Comparison to SAS 9 Authorization Admin Notebook: Folder Security – SAS Viya vs. SAS 9 An Introduction to Authorization in Viya Macros for exporting permissions. I would like to thank my colleague David Stern for his valuable input to this post. ... View more

In a previous post, I talked about updating SAS Viya. In this one I will look at upgrades. An upgrade adds significant features, changes, or improvements to your deployed software. An upgrade is performed to move from one version of Viya to another, for example from 3.3 to 3.4. The process to do an upgrade is similar to an update. In this post I will look at upgrading from Viya 3.3 to Viya 3.4   NOTE: A change in terminology occurred with the release of Viya 3.4 second edition. This release introduced significant new functionality as a result you need to generate a new order to move an existing Viya 3.4 version to the 3.4 second edition. To quote the documentation “there are two types of upgrade associated with SAS Viya 3.4 software. You can upgrade products in SAS Viya 3.4, such as SAS Econometrics 8.3 to SAS Econometrics 8.4. You can also upgrade to SAS Viya 3.4 from earlier versions of SAS Viya, such as SAS Viya 3.2 or SAS Viya 3.3.” The process for upgrading products within Viya 3.4 has its own section in the documentation “Adding SAS Viya Software to a Deployment and Upgrading Products in SAS Viya 3.4” This blog was written prior to the second edition relase and deals with upgrading from earlier versions of Viya (3.1, 3.2, 3.3) to Viya 3.4.   Upgrade to Viya 3.4 is supported from 3.2 and 3.3 and additional software can be added to the deployment during an upgrade. Upgrade is designed to maintain all content created in the previous version of Viya, so any reports, data plans or models will be preserved.   Step 1 - Plan your Upgrade You will need a new software order to upgrade your deployed software and an upgrade will most likely require changes to the deployed software’s configuration. You will need to plan for some downtime, upgrading SAS Viya software will require an outage period because SAS Viya services are stopped and restarted automatically during the upgrade process.   You should create a full system backup of your deployment prior to performing an upgrade, there is no rollback from an upgrade, failed or not, so a good backup is critical. In addition, you should also make sure the environment you are upgrading meets all the system requirements for the new version of the software. System requirements do change between releases.   Check the documentation for any manuals steps required to be performed in the current environment prior to the upgrade.   Step 2 - Prepare for the Upgrade Merge Ansible Files From your new software order, using the orchestration cli, generate a new playbook and store it in a new location. Make sure you keep a copy of the old playbook, you will need it. Now that you have your new playbook you need to merge the settings from your existing playbook into the new playbook. You must do this to preserve the customizations you made when you deployed the software, and to include any new default settings provided by your new order.   The two important files are: vars.yml which contains deployment variables that enable you to customize your deployment to meet your needs. inventory.ini which specifies the machines to be included in a deployment and the software to be installed on them. First, let us look at vars.yml. You can compare the files in a way you are comfortable with. You can use the LINUX diff command, or there is s a nice Linux utility called Meld. There is a Windows version of Meld also, or you can use other Windows tools like WinMerge.   We do the comparison so that we can maintain any customizations you made in the original deployment, and also set values for any new variables in the new vars.yml.   The documentation recommends you use three files in the comparison. vars_original.yml the 3.3 vars.yml before any edits were made. vars_current.yml the 3.3 vars.yml used to deploy the software vars.yml the new 3.4 file for the upgrade There is no automated way to merge the files. Whatever tools you use, you will still have to go item by item through the comparison and decide what to do in each case. The decisions you make are based on your knowledge of the existing deployment, and of the requirements for the upgraded deployment.   Comparing the vars_current.yml (used to deploy 3.3) with the new vars.yml (before any customizations are made).   diff /root/sas_viya_playbook_33/vars_current.yml /root/sas_viya_playbook/vars.yml   When comparing the variables and values in the old and new vars.yml files a number of scenarios can occur: DEPRECATED: In vars_current.yml and not vars.yml: deprecated variables, these should not be brought forward to the new customized vars.yml for 3.4 NEW: In the new vars.yml for 3,4 but not in 3.3 file (vars_current.yml). New settings for 3.4 set appropriately When there are existing variables with different values in the two files, the person doing the upgrade has to make an intelligent decision to update the values for the variable. The choices could be: EXISTING-KEEPOLD keep the value from 3.3 EXISTING-KEEPNEW keep the new default value EXISTING-MERGE some variables are more complex and will have to be merged. Examples always help, right? Each row in the table below has a variable and the scenario for that variable. There are the merge decisions made when upgrading the GEL Administration environment.     The rest of the changes in my vars.yml were the addition of comments and other optional changes.   Next we need to merge the inventory files. To do that compare the old and new inventory.ini files. Ansible uses an inventory file to specify the machines to be included in a deployment and the software to be installed on them.   Each inventory file consists of two parts: deployment target definition : a specification of each machine on which SAS Viya software will be deployed host group assignment which lists a mapping of the installable groups of software and the machines on which they will be deployed. The deployment target definition is likely to be the same in the upgraded environment, so you can copy that section of the file. However, host groups will probably change with new ones being added, and potentially old ones being removed.   This picture shows the start of the comparison of the old file (left) to the new file using meld.   meld "/root/sas_viya_playbook_33/inventory.ini" "/root/sas_viya_playbook/inventory.ini"     You can use the arrows in meld to implement the changes in the new inventory.ini file. The tool can help you make these changes but you need to be careful during the process and check what you are doing. For example, in the screenshot below if the arrow was used to make the change the StudioViya host group would be removed from the inventory.ini.     When editing the host groups make the following changes. Where a host group exists in the new inventory file make sure you allocate the same machines as you had in the old inventory file. If a new host group has been added, consider where it should be placed and add that machine to its host group. In my comparison I had two new host groups StudioViya and GraphBuilderServices. Keep any comments that have been added to the new inventory file. Note: Because of a change in the structure of the host groups between releases, ensure that the inventory.ini file for the upgrade also includes the machine in the [Operations] host group in the [CommandLine] host group.   You should also make sure you transition any changes you made in the ansible,cfg file to the new ansible.cfg file.   The next step is to record the current versions of your software. This is a process I discussed in my previous article where you can find the detailed steps. In this article I am not going to detail that process again.   Manual Preparation Steps The deployment guide section on upgrading lists quite a few manual steps that need to be performed to prepare for an upgrade. Read the document carefully to determine what manual steps are required in your environment. The steps vary based on the whether your source environment is 3.2 or 3.3 and also what software you have licensed and how it was configured (HA, multi-tenant etc). A few of the more common tasks are listed here and the details of how to do them are in the deployment guide: Delete the Default Backup Schedule Job Verify the Health of the SAS Infrastructure Data Server Remove Repositories maintain continuity for auditing after the upgrade (Prepare the [Operations] Host Group for Upgrade) A final step before the upgrade. Run the system-assessment.yml too assess the readiness of your system for deployment of the new software. Prior to deploying fix any issues that are uncovered.   Step 3 - Upgrade With all the preparation done the actual upgrade should now be simple. To upgrade the software use the same ansible command that was used to deploy the software. In many cases this step will stop and restart Viya services, as result it is recommended that an outage is scheduled during the upgrade.   ansible-playbook site.yml -i inventory.ini   Step 4 - Post Upgrade Manual Steps Similar to the pre-upgrade manual steps there may be post upgrade manual steps. Check the deployment guide to determine which ones apply to your deployment.   A common post upgrade manual step is the need to deregister obsolete services. As part of the upgrade process, several services from your old SAS Viya environment will be removed or replaced with other microservices. It is possible that these obsolete microservices might not be fully deregistered from Consul. The Consul log will repeatedly record messages about these failing services. In addition, you will see old instances of the obsolete services in the dashboard and machines area of SAS Environment Manager. This is because they are still registered in consul. Complete the deregistration steps to remove the services.     An additional manual step, listed in the upgrade section that is required is to run a script(add_new_caslib_controls.sh) which makes required changes to permissions on Viya CASLIB's to ensure that internal accounts have the correct level of access.   The documentation also recommends you run an immediate backup of the system. This will be your baseline backup on Viya 3.4. Any backups made prior to the upgrade should not be used and can be deleted.   Step 5 - Test, Test, Test Ok, your ansible playbook ran, your manual steps are completed. You are not done, the deployment needs to be thoroughly tested. You should also test any existing content such are reports, plans, projects or models. In addition you will need some education of your user community as there are some significant changes with Viya 3.4. Users will notice immediately that SASHome is gone and has been replaced by SASDrive.   The upgrade process for SAS Viya does require some attention to detail. There are quite a few manual steps both prior to and after the upgrade. If you pay attention to the details and plan and prepare for the process, you can successfully upgrade to the latest version of SAS Viya.   This article was a team effort. I would like to thank Erwan Granger, Cindy Taylor, Dawn Schrader, Connie Robison, and Jawna Gardner for their invaluable input. ... View more

With SAS Viya the concept of hot fixes established in previous versions of SAS will continue. SAS provides hot fixes to previously shipped software. A hot fix is created to resolve a number problems in released versions of our software. So, with the adoption of SAS Viya increasing, and the upcoming release of Viya 3.4, it seems like a good time to talk about upgrades and updates in the Viya world.   Firstly, lets look at terminology, in Viya the terms update and upgrade are related but different.   You can update your deployed software environment in order to get the environment’s software to the latest version. As an example, when hot fixes are released for a version of Viya, such as 3.3, you would update your software to get the hot fixes and any other updates to the SAS executables. An update does not require a new software order.   An upgrade adds significant feature changes or improvements to your deployed software. An upgrade is performed to move from one version of Viya to another, for example from 3.3 to 3.4. You will need a new software order to upgrade your deployed software, and an upgrade will most likely require changes to the deployed software’s configuration.   In the first of two posts on this topic I will look at the simpler scenario, updates. One of the most common reasons for doing an update is the availability of a hot fix. The method of staying informed about available hot fixes has not changed from SAS 9.4. For more details see Chris Hemedinger's article on how to stay informed about SAS hot fixes.   I recently updated the deployment we use for the administration section of the GEL Viya Implementers workshop to apply some SAS Environment Manager hot fixes. If you were used to applying hotfixes in SAS 9.4 be prepared for some changes. Unlike SAS 9.4 to apply a hot fix you do not need to specifically download a package of the hotfix. Since SAS Viya software is delivered via RPM's then the same process that was used to install the software can be used to apply updates.   The update process brings your deployed software up-to-date with the latest compatible software. You perform the update with the same command that was used to install SAS Viya, and you use the same software order and the same playbook. Running the playbook updates all software to the latest version (no partial updates). To perform the update process, you must have administrator privileges and sudo access for the machines.   Something to be aware of in relation to Viya is that the nature of the process, where updates are performed with yum via ansible, makes it much easier to do an accidental update. A yum update will get you the latest software, for this reason setting up a mirror repository is a good practice.   The Viya 3,3 deployment that I updated has the Viya services split across 4 separate machines and has a single SMP CAS server. In this scenario, where we need to run commands on each machine in the deployment, ansible is your friend. The process involves comparing the RPM versions on each machine on the deployment before and after the update is installed. To automate this part of the process we created two ensile playbooks.   gel.compare.update.before.yml: stores the state of the SAS Viya RPM's and repositories prior to the update gel.compare.update.after.yml: stores the state of the SAS Viya RPM's after the update and compares them to the before state. The full process to apply the hot fix is as follows (the storing and comparing rpm states can be automated using the ansible playbooks).   Record the current version of the RPM packages installed   The first step in the process is to record the version of the RPM files currently installed so that we can use them to compare the current deployment to the updated deployment at the end of the process. To do this execute yum list installed on each machine. The command produces a list of all packages installed on each machine that include the string "sas-" in the name of the repository.   yum list installed | sort | grep sas- > /tmp/yum.before.update.txt   The output shown below lists package name, version and release date, and the repository.     You can also use the rpm -qa command to list installed packages.   rpm -qa | sort | grep sas- > /tmp/rpm.before.update.txt   The output of the rpm command is shown below.      Check for user updates in _deployment files   Any Configuration files that include the string _deployment may be overwritten by the update process. These files should not have been updated, however if they have updates, they should be identified and saved for re-implementation. Use the following command to identity all the "*_deployment" files and check them for any changes made by users.   ansible sas-all -b -m shell -a "find /opt/sas/viya -name '*_deployment*' 2> /dev/null -exec ls -al {} \; | grep '_deployment'"   The output below shows a partial list from one of the Viya machines.     Update the mirror   If you are using a mirrored repository you need to re synchronize the mirror prior to performing the update. The re synchronize makes the new software available to the mirrored repository so that it can be installed in the environment.   Use ansible to run the update and apply the hot-fix To update the software and apply the hot-fixes use the same ansible command that was used to deploy the software. In many cases this step will stop and restart Viya services, as result it is recommended that an outage is scheduled during the update.   ansible-playbook site.yml -i inventory.ini -e "@repo_override.txt"   Record the updated versions of the RPM packages For comparison purposes record the versions of the RPM's after the update.   yum list installed | sort | grep sas- > /tmp/yum.after.update.txt     rpm -qa | sort | grep sas- > /tmp/rpm.after.update.txt     Compare the before and after for the RPM'S The playbook runs the commands above on each machine, concatenates all the files and then compares the state of the deployments before and after the update using the diff command.   diff /tmp/yum.before.update.txt /tmp/yum.after.update.txt > /tmp/diff.txt     The output of the diff command comparing the before and after state of the RPM's shows that (among other changes) the SAS Environment Manager RPM versions were updated from version 1.3.31 to 1.3.34 with the date changing from November 20017 to March 2018, the date the hot fix was released.   The diff command shows what files have been changed by the application of the update. It would be nice to see from the hot fix documentation if that matches the expected changes based on the update that is being applied. This would make the validation more robust. Maybe in a later release.   Before you start this process I strongly recommend two things:   Create a full backup of your Viya Deployment Develop a test plan which you will execute to determine the success of the update.   When the process is completed execute your test plan, testing access to applications and any content that has been developed in the environment.   The documentation for the update process can be found in the SAS Viya 3.3 Administration guide. Overall the update process was relatively painless for me. I did have a lot of help though. Thank you Erwan Granger and Gilles Chrzaszcz. ... View more

In the REST based architecture of Viya, the Uniform Resource Identifier (URI) is how we define and identify resources within the system. Understanding the URI is important for administrators. URI’s are used to define authorization rules governing the privileges users have within the Viya system. In addition, the URI is often required when using the command-line interfaces, or REST API, to identity unique resources. Some aspects of the URI and its syntax are confusing. In this post, I will try to explain the format and function of URI’s in SAS Viya.   What is a URI? Wikipedia says: “in information technology, a Uniform Resource Identifier (URI) is a string of characters used to identify a resource.” Exactly! Each resource in Viya is identified by a URI and can be either content (aka objects) or functions surfaced by applications and services. When we want to find content to view, modify or update the URI is how we know we are looking at the correct item. When we want to define functionality, the URI is how we refer to the application or service facilitating those functions. In Viya a resource could be content, e.g. a folder or a report; or application, service or group of functions.   Content URI's A Content URI refers to an object in Viya such as a report, data plan or project. Content URI's typically have this format:   /servicename/endpoint/uniqueid   Below are the basic properties for a report, including its URI which includes its globally unique ID or GUID. name: Sales Correlation URI: /reports/reports/c997e5a2-7ccb-4552-b1a5-d8b0a3cb1af6 location: /gelcontent/GELCorp/Sales/Reports/Sales Correlation     The report is named Sales Correlation and is located in the folder /gelcontent/GELCorp/Sales/Reports/Sales Correlation. The URI is how the report is uniquely identified. We can rename the report and/or move it to another folder within the Viya folder structure and it will retain its unique URI.   The fact that the URI starts with reports twice(/reports/reports) is coincidental; in this case the servicename and endpoint have the same name. If we were to look at a Data Plan, for example, this would not be the case. It would be something like /dataPlans/plans/22bbaeae-63ea-4d6b-a809-0e10f7faafdc.   Application/Functionality URI's URI's which relate to applications or functionality  become important when an administrator wishes to allow or restrict access to functions and features as I discussed in this blog. When it comes to functionality there are a number of different types of URI.   Service: Micro-services and Web Applications have this URI format:   /<service_name>.   An example of a service URI would be /SASEnvironmentManager or /SASDataExplorer. In terms of permissions If you have the permission READ for a service, you are allowed to use that service/application. If a user or group were denied READ permissions to /SASEnvironment Manager/** then they would not be able to access the application.   Service endpoints: are functionality within services; the URI format is:   /<service_name>/<end_point>/<end_point>….   This represents a set of descendent functionality.  For example, /SASEnvironmentManager/dashboard would refer to just the dashboard within SAS Environment Manager.   Psuedo URI’s were created to support functionality that doesn’t correspond to a single endpoint. They help define authorization rules, using a single URI, for some privilege that would otherwise require a group of rules to characterize over multiple endpoints and permissions. Essentially they represent application capabilities which do not map to a specific endpoint. A psuedo URI has this format:   <service_name>_capabilities/<capability>.   An example of a pseudo URI would be /webDataAccess_capabilities/googleanalyticsImport or /SASVisualAnalytics_capabilities/buildAnalyticalModel. The latter refers to the group of menu choices which allow a user to build analytic models within Visual Analytics.   Often a URI in Viya will haves /** at the end of some of them. The /** are wildcards that apply to the descendent functionality of an endpoint. For example, /SASEnvironmentManager/** means the SAS Environment Manager service/application and all of its descendent functionality. An asterisk /* refers to one additional level of functionality. The single * can also be found in the middle of a URI.   What about when you see a folder or content object with /** at the end? You may incorrectly assume that this refers to the folder and its content, like reports and sub-folders. This is not true, the ** has nothing to do with content. Instead it means, to quote the Viya Administration documentation, "access to the associated service’s endpoints in the context of that particular object". In other words if, for example, we have the following folder URI:   /folders/folders/e52e24a0-193d-479f-9261-6134d19300e5/**   The ** means that, in the context of the folder, the URI refers to all the actions that can be performed on the folder by the folders micro-service and folders endpoint of that micro-service. If we look at the micro-service documentation for folders, these actions would be things like, listing, adding and deleting members.   As I mentioned earlier, there is some complexity to understanding the URI in SAS Viya, I hope that this post has helped improve your understanding. ... View more