Viya Inventory CLI and Reports
The critical stage in any migration or promotion exercise is the planning stage. During planning you need to understand the source environment in detail. This understanding helps you to identify all the items you will need to move to your target system, determine if you are ready to proceed and estimate the resources that will be required. To do all this you need help! New in SAS Viya 3.5 the inventory CLI and reports are designed to help in the critical planning stage of migrations and promotions to new versions of Viya.
The Viya inventory CLI will help you:
- Locate and record all Viya content
- Document Space requirements for the target system
- Determine readiness.
There are two steps involved in using the tool: scanning and publishing.
Scanning
There are four types of scan:
- Services scans content stored in infrastructure data server and inventories artifacts from each service.
- CAS scans CAS servers, libraries and tables, permissions and configuration.
- File System scans the file system default locations for Viya created artifacts.
- Infrastructure scans for disk size of artifacts in the Viya system.
The scans output results to .csv and .yml files in a directory on the file system.
In the publish step the csv files are aggregated into a master csv file that can be loaded to CAS.
The diagram below from the SAS Viya 3.5 Administration guide shows an overview of the process.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
The inventory CLI has certain requirements that must be satisfied so that it can access all the resources it needs. You must be aware of:
- what machine(s) each scan must run on
- the host account you use to run the CLI
- the account you use to authenticate to Viya.
The account you use to authenticate to Viya must always be an account that is a member of the Administrators group in Viya.
The table below summarizes the requirements for each type of scan. (The names in brackets [ ] refer to the host group in the inventory.ini file used to deploy the software).
Note: if two requirements for a type of scan are met on one machine the scan need only be run once on that machine. For example if the Infrastructure Data Server and Consul are on one machine then infrastructure scan only needs to be run one on that machine.
If you have access to the root account, or a user that can sudo, you can simplify your life and run everything as root. In addition, if the cas account is a member of the same group as the sas account then all scans with the exception of the infrastructure scan can be run using the cas user.
The steps to run each scan are all very similar. You must authenticate to Viya as an administrator as you do for all sas-admin CLI operations, then you can run the scan. The example below shows the steps to run the scan in our GEL Viya administration environment.
There are three parameters to sas-admin inventory scan.
- customer-id: the name of the customer or overall environment
- deployment-label: a name for the deployment you are scanning. For example, dev, test or prod.
- output-location for the initial scan results.
The scan result csv files are named in the following pattern:
SASViyaInventory_scan-type_deployment-label_short-host-name.csv
For example: SASViyaInventory_FileSystem_prod_intcas01.csv
Each run of a scan will replace any files that exist in the directory that have the same deployment label. Before the publish step all .csv files from all of the scans must be in the same directory. It is helpful to have some shared directories that each machine can access to store the initial scan results and for the publish location.
Publishing
Publish aggregates all csv files from the scan output location and creates two master files SASViyaInventory.csv and SASViyaTypes.csv . Publishing will overwrite the existing SASViyaInventory.csv and SASViyaTypes.csv
/opt/sas/viya/home/bin/sas-admin inventory publish --source-location /tmp/inventory_all --output-location /gelcontent/gelcorp/inventory
The final step is to make the data available to CAS. The data must Load to CAS in the System Data CAS Library for use by the Inventory report. There are a number of ways you can achieve this:
- Publish directly to the System Data caslib source
- Copy from the publish location System Data caslib source
- Import to the System Data Caslib using
- SAS Environment Manager
- sas-admin cas CLI
- CAS Table State Management Jobs
The results of the scan are visualized in the SAS Environment Manager Inventory Report accessible from the SAS Environment Manager Dashboard in SAS Viya 3.5.
There is not enough room here to describe all the details in the reports. Some of the highlights are:
The What’s being used tab gives an overview of the software usage by either SAS product totals (Content Items Found) or more specific totals of SAS objects (Types of Items Found), SAS servers (Server, Providers and Contexts), and SAS storage (Data Item Storage). You can also find details of the type and volume of content in the Object Inventory tab.
The CAS Inventory tab provides details on all the caslib's in the environment.
The graph in the report ranks the Caslibs from largest to smallest in terms of storage. Selecting a caslib will display its tables and their size.
The New System Provisioning tab. This data can help you understand what resources will be required to migrate to a new Viya system. It shows the size of the
- Infrastructure Area (postgres and consul data size)
- Data Areas (caslibs)
- Non-Data File system areas (configuration)
While the inventory CLI started life as a tool to help with migration and promotion, it has the potential for many other uses in the areas of monitoring, auditing and troubleshooting. Going forward the plan is for the tool to feed into the process of migrating to new versions of Viya. Future development plans do change, but the current plan is for the tool to support identifying and gathering the critical content and configuration from a Viya deployment to support migration to the next Viya release which, as we all know, will be on K8s and containers.
I hope you found this useful. For more information:
