Administering your models programmatically in SAS Viya
In my last article, we looked at using REST to interact with the SAS Viya platform to view audit records. Shortly after, I faced another challenge where I thought REST might be able to help me. I needed a way to administer models, model projects and model repositories from the command line. For instance, I needed a way to list all model projects, and delete a model programmatically. As of the time of writing, the sas-admin CLI does not have a models plugin or an equivalent that offers this kind of functionality for models. In this post, we'll look at how tasks like these can be done using various tools available in Viya.
Overview
My first hurdle was understanding the different model artefact types (I was a model idiot). Thankfully, my GEL counterpart @XavierBizoux helped me make sense of it. A model repository is a dedicated location where projects and models can reside. An environment may have multiple model repos to provide logical separation (e.g. for Dev & Test, or by business unit, etc.). So what about model projects? Here's where it can get a little confusing. In Model Manager, a project is similar to a folder that may contain one or more models. In Model Studio (included with VDMML), a project is an artifact/object type, wherein models can be built, compared, registered and published. The two are not alike, and model projects created in Model Studio do not appear as 'projects' in Model Manager and vice versa (unless a model is registered in Model Studio, in which case it will appear in Model Manager inside a project with the same name as the Model Studio project).
Now for my use case. To automate and simplify administration tasks, I wanted to programmatically:
- list existing models and projects
- delete models and projects
- perform other administrative tasks (such as create model repositories)
The sas-admin CLI
My first port of call was the sas-admin CLI. In the absence of a models plug-in, I looked to the folders plug-in. We can see reports inside folders if we use the folders list-members command, and the same is true for models and model projects. We can also add --recursive or --tree to list all contents of folders recursively. /opt/sas/viya/home/bin/sas-admin --output text folders list-members --path "/Model Repositories/DMRepository" --recursive
Id Name Type Description Uri
e92c01e1-1ada-4b52-8157-cba93c616cb9 My_MS_project child /folders/folders/463af6e3-e827-42e9-835c-f99aef6bad63
48e8ceed-1470-485d-9533-938429434edf My_MS_project child /modelRepository/projects/75df5ca4-8ce1-4d60-b46e-daebfeecec0b
b7edcffa-16ef-4de1-a6a4-9b18076fa71a Version 1 child /folders/folders/ddab2cfc-4f43-4021-b92a-73f802ace9b0
8bc578a7-f4bf-4008-880e-79f8aa682a4b Logistic Regression (Pipeline 1) child /folders/folders/6df8f34b-ea5d-49ac-a507-2e9e3d9c0f36
e52ac6e8-2edf-4ff0-9a1b-517fb6a5908e Logistic Regression (Pipeline 1) child /modelRepository/models/d4bdf81e-9877-4683-bbda-9204c6e5a5c9
What happens if we try to delete a folder that contains models using the CLI? For the most part, it works. If we add the --recursive flag to the folders delete command, it deletes models and projects in the specified folder, although we are presented with an error message.
/opt/sas/viya/home/bin/sas-admin --output text folders delete --path "/Model Repositories/Sales" --recursive
Are you sure you want to delete this folder? Specify "y" or "yes" to delete.> y
The following errors have occurred:
1 error occurred:
* The requested resource could not be found.
Http Status: 0/code>
A quick check shows that the contents of the folder have been deleted, but not the folder itself. One thing to remember though, is that in this example, the Sales folder is our repository location. Repositories cannot be deleted until all of their contents are deleted. If we simply run the same command again, the folder itself (and the repository) will be deleted.
Other tasks such as creating repos and performing project tasks cannot yet be done using the CLI. For those kind of actions, we need to look at some of other available options.
Model Manager macros
Model Manager offers a number of macros, which provide the type of functionality I was after. Using these macros in a SAS program, it's possible to create Model Manager projects and repositories. Let's look at creating a project as an example. In SAS Studio, we can execute the following program.
%let servernm=http://sasviya01.race.sas.com;
%let userID=geladm;
%let password=lnxsas;
%mm_get_token(
baseURL=&servernm,
user=&userID,
pw=&password,
tokenname=myTokenName
);
%mm_get_repository_id(
repositorynm=DMRepository,
idvar=myRepID,
servernm=&servernm,
token=%myTokenName
);
%mm_create_folder(
foldernm = MyMarketingProjectFolder,
reposfolderID = %str(&_fldrID),
servernm = &servernm,
token = %myTokenName
);
%mm_create_project(
projectnm = MyMarketingProject,
folderID = %str(&_folderID),
function = Classification,
servernm = &servernm,
token = %myTokenName
);
In this program, we first set the necessary parameters, then we get the ID of the repository we want to create our project in, we create a folder for the project and then we create the project itself. We can log on to Model Manager to see the result.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
This simple example demonstrates how the Model Manager macros can be a handy resource for administering models. In addition to creating projects and repositories, we can also delete models, import models and perform various other actions using only SAS code.
REST and pyviyatools
Model artefacts can be surfaced by calling the /modelRepository/ REST service endpoint. Using other, existing pyviyatools as examples, I wrote a simple Python program to query existing models, projects and repositories by calling that endpoint (using callrestapi.py). listmodelobjects.py has been added to the pyviyatools repository and lists all models when run with no arguments by calling the /modelRepository/models endpoint: ./listmodelobjects.py
-------------------------------------
Listing model objects
-------------------------------------
id ,name ,type ,description ,creationTimeStamp ,modifiedTimeStamp
"d4bdf81e-9877-4683-bbda-9204c6e5a5c9","Logistic Regression (Pipeline 1)","None","None","2020-02-20T07:52:24.614Z","2020-02-20T07:52:30.693Z"
but can also list model projects (/modelRepository/projects endpoint) and repositories (/modelRepository/repositories endpoint). ./listmodelobjects.py -c project
-------------------------------------
Listing project objects
-------------------------------------
id ,name ,type ,description ,creationTimeStamp ,modifiedTimeStamp
id ,name ,type ,description ,creationTimeStamp ,modifiedTimeStamp
"f35f11eb-984e-4feb-9c20-3ea896fc5c0e","MyMarketingProject","None","None","2020-02-26T03:01:16.749Z","2020-02-26T03:01:16.995Z"
"75df5ca4-8ce1-4d60-b46e-daebfeecec0b","My_MS_project","None","None","2020-02-20T07:52:22.069Z","2020-02-20T07:52:30.611Z"
The complete usage pattern is displayed in the built-in documentation.
./listmodelobjects.py -h
usage: listmodelobjects.py [-h] [-n NAME] [-c TYPE] [-d DAYS] [-m MODIFIEDBY]
[-s SORTBY] [-o {csv,json,simple}]
optional arguments:
-h, --help show this help message and exit
-n NAME, --name NAME Name contains
-c TYPE, --type TYPE Content Type (model, project or repository)
-d DAYS, --days DAYS List files older than this number of days
-m MODIFIEDBY, --modifiedby MODIFIEDBY
Last modified id equals
-s SORTBY, --sortby SORTBY
Sort the output descending by this field
-o {csv,json,simple}, --output {csv,json,simple}
Output Style
To delete models, there are a couple of things to consider.
First, we need appropriate permission to delete objects.
Second, models can be deleted by passing in their URI to the REST call. The output from the listmodelobjects.py program does provide the ID, which can be used to construct the URI. For instance, we can write a program to delete the MyMarketingProject project we created with the macro in the previous step by referencing its URI.
import argparse , datetime, sys
from sharedfunctions import callrestapi,printresult,getfolderid,getidsanduris
from datetime import datetime as dt, timedelta as td
# prepare the delete request
reqtype='delete'
reqval="/modelRepository/projects/cdb88b35-7634-407f-b959-52a0e13ecac3"
# Make REST call to delete project
files_result_json=callrestapi(reqval,reqtype)
This works, but a nicer option would be to refer to models or model projects by name. To do this, we can query the JSON object returned by a GET call to /modelRepository/projects in our code using jq to extract the ID. So, our updated code would look like the following:
# Extract ID using jq
projectid=$($pyviyadir/callrestapi.py -m get -e "/modelRepository/projects?filter=eq(name,'MyMarketingProject')" | jq -r '.items[]["id"]')
# Make REST call to delete
$pyviyadir/callrestapi.py -m delete -e "/modelRepository/projects/$projectid
Note that while model projects must have unique names, there can be more than one model with the same name. Be careful not to delete the wrong model!
Once the objects within a repository are deleted, we can also delete the repo itself by modifying the code sample above to point to /modelRepository/repositories instead of /modelRepository/projects and specifying the repo name.
We can also create repositories, and perform other, more advanced tasks on models using REST calls, such as setting a model as champion, or updating model metadata. Refer to the externally-facing SAS Developer site for a listing of the public REST & CAS APIs, as well as downloadable code libraries, modules and documentation for developers.
Acknowledgements
My thanks to my GEL colleagues for their insights and contributions. Thank you for reading. I hope the information provided in this post has been helpful. Please leave a comment below to ask questions or share your own experiences.
