Install and use the pyviyatools on Windows

[#1]

In this Part 2 of a two-part series, we will install and use the pyviyatools on Windows.

The pyviyatools require the SAS Viya CLI and the trusted certificates file it requires, and of course Python - preferably python3. In this guide I chose to install pyviyatools using git from a command line, though like most of the things featured in this guide, there are other ways to get it. If you followed Part 1 of this guide, Install and use the SAS Viya CLI on Windows, much of the hard work is already done.

If you have not already read and followed Part 1, I would suggest you glance at this guide to get an idea of what it covers, and then go and follow the steps in Part 1 first.

This guide documents the concluding part of one possible ‘happy path’ for setting the SAS Viya CLI and pyviyatools up on Windows from scratch. If you have some of the components already, you may find you can skip the corresponding section. But between Part 1 of this guide and the steps below, below you can find the complete process.

Contents

In Part 2 of this guide, we will cover:

• Dependencies
• Install pyviyatools and the packages it requires
  • Python3
  • Upgrade pip
  • Python3 requests module
  • Python3 configobj module
  • git
  • pyviyatools
• Use pyviyatools from a Windows command prompt
  • Create a .authinfo file
  • Authenticate using loginviauthinfo.py and your .authinfo file
  • pyviyatools Examples
    
    

•  

No doubt this guide could be improved. If you see areas you think are overcomplicated, wrong or unnecessary, or if you have questions about why I chose a particular command or installation method, I would love to hear from you. Leave a comment below or get in touch!

Dependencies

In this Part 2, we assume all the components from Part 1 are already present, so we only need to install the things in the 'Part 2' box in the dependency diagram. We will assume that everything outside that box (the Windows client machine, chocolatey, the SAS Viya CLI and the trustedcerts.pem certificates file which the pyviyatools also requires) are already set up and validated: 

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Install pyviyatools and the packages it requires

Before proceeding with these steps, ensure you have followed the steps in Part 1 of this guide, and that the SAS Viya CLI is working from a Windows Command Prompt on your Windows machine, that it is connected to your SAS Viya deployment, and that you can authenticate successfully with a SAS Viya username and password and can run SAS Viya CLI commands successfully. To set up pyviyatools and the things it requires, we return to using PowerShell.

Python3

There are many ways to install Python on a Windows machine, and any of them should be okay if you can run python --version, from both PowerShell and Windows Command Prompt and get a Python 3.x.x version number in response. For the sake of consistency, we will install it using chocolatey here.

1. Open a new PowerShell terminal window, and see if Python3 is already installed:

python --version

If Python3 is installed, you should see something like this, and you can continue to the next section, to upgrade pip:

Python 3.11.4

If Python3 is not installed, you should see something like this (in red):

python3 : The term 'python3' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ python3 --version
+ ~~~~~~~
    + CategoryInfo          : ObjectNotFound: (python3:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

2. If Python3 is not installed, run this command to install Python3 using chocolatey, which takes a few minutes:

choco install python3 -y

3. After installing python, close the PowerShell terminal, and open a new one.
4. Then, run python --version again to verify that Python3 is installed and in your path. You should see something like this:

Python 3.11.4

Upgrade pip

 

1. From a PowerShell terminal, run this to upgrade the Python package manager, pip:

python.exe -m pip install --upgrade pip

Python3 requests module

 

1. From a PowerShell terminal, run this to install the Python3 requests module, a rich set of tools for making HTTP requests, e.g. for calling REST APIs like those exposed by SAS Viya:

python -m pip install requests

Python3 configobj module

 

1. From a PowerShell terminal, run this to install the Python3 configobj module, a config file reader and writer:

python -m pip install configobj

 

git

Install git if it is not already installed. Git can be installed many ways; any method is probably fine so long as you end up getting a response similar to that shown below when you run git version. Since we are using chocolatey to install other packages in these instructions, we may as well pick that method for this step.

1. See whether git is already installed. In the PowerShell terminal window, run this:

git version

If you see something like this, git is already installed:

git version 2.41.0.windows.1

If you see something like this, git is not installed:

git : The term 'git' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ git
+ ~~~
    + CategoryInfo          : ObjectNotFound: (git:String) [], CommandNotFoundException

2. If git is not already installed, run this to install it:

choco install git.install -y

3. After installing git, close the PowerShell terminal, and open a new one.
4. Then, run git version again to verify that git is installed and in your path. You should see something like this:

git version 2.41.0.windows.1

pyviyatools

We are finally ready to deploy pyviyatools.

1. Clone the pyviyatools project from GitHub into C:\ViyaHome, using git. pyviyatools is a public project, so you do not need to authenticate to GitHub. Run this from the PowerShell terminal:

Push-Location
Set-Location Env:
cd C:\ViyaHome
git clone https://github.com/sassoftware/pyviyatools.git
Pop-Location

2. Verify that the pyviyatools project was cloned to C:\ViyaHome\pyviyatools:

dir C:\ViyaHome\pyviyatools

Partial expected output (with some files omitted):

PS C:\Users\student> dir C:\ViyaHome\pyviyatools


    Directory: C:\ViyaHome\pyviyatools


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----        7/14/2023   6:24 AM                templates
-a----        7/14/2023   6:24 AM             57 .gitattributes
-a----        7/14/2023   6:24 AM            153 .gitignore
-a----        7/14/2023   6:24 AM             70 application.properties
-a----        7/14/2023   6:24 AM           6521 applyauthmodel.py
-a----        7/14/2023   6:24 AM           4857 applyfolderauthorization.py
-a----        7/14/2023   6:24 AM           4308 applyviyarules.py
-a----        7/14/2023   6:24 AM           7899 archivefiles.py
-a----        7/14/2023   6:24 AM           3656 callrestapi.py
...
Some files omitted
...
-a----        7/14/2023   6:24 AM           3987 toggleviyarules.py
-a----        7/14/2023   6:24 AM           5327 unittestsadm33.sh
-a----        7/14/2023   6:24 AM           8687 unittestsadm34.sh
-a----        7/14/2023   6:24 AM           4563 unittestviya4.sh
-a----        7/14/2023   6:24 AM           3150 updatedomain.py
-a----        7/14/2023   6:24 AM           4731 updatepreferences.py
-a----        7/14/2023   6:24 AM           7843 validateviya-manual.md
-a----        7/14/2023   6:24 AM          23618 validateviya.py


PS C:\Users\student>

3. Run the pyviyatools setup.py script. This tool uses the Python3 configobj module we installed earlier to set values in a pyviyatools configuration file called application.properties. In the PowerShell terminal window, run this, substituting the location where you installed the sas-viya CLI executable for C:\ViyaHome\CLI, if it is somewhere else on your Windows machine. Recall that we installed and configured the SAS Viya CLI in Part 1 of this guide.

cd C:\ViyaHome\pyviyatools
./setup.py --clilocation C:\ViyaHome\CLI --cliexecutable sas-viya

Expected output:

Note: updating C:\ViyaHome\pyviyatools\application.properties
NOTE: configuration values set to  {'sascli.location': 'C:\\ViyaHome\\CLI', 'sascli.executable': 'sas-viya'}

4. Optional: If you want to be able to run pyviyatools without specifying the path, add the pyviyatools directory to your path. In the PowerShell terminal window, run this:

[Environment]::SetEnvironmentVariable("Path",[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + ";C:\ViyaHome\pyviyatools", [EnvironmentVariableTarget]::Machine)

5. Close the PowerShell terminal window, and open a new one, to pick up the change to the machine’s path environment variable.
   
   

Use pyviyatools from a Windows command prompt

 

Create a .authinfo file

 

1. Create a .authinfo file. In the following command, I have used the username and password of a SAS Viya administrator account: user geladm, password lnxsas. Substitute the username and password of the account you wish to use by default for these values in the command below. In the PowerShell terminal window, run this

$authinfo_file=$HOME+'\.authinfo'
[IO.File]::WriteAllLines($authinfo_file, 'default user geladm password lnxsas')
type $authinfo_file

Note: We use WriteAllLines here to write the line to the .authinfo file, because when I used echo, the character encoding ended up being UTF-16, which was not understood by python3’s configObj module.

Authenticate using loginviauthinfo.py and your .authinfo file

I have not had success getting this working from a PowerShell prompt yet, so here we switch to a Windows command prompt.

1. Open a new Windows Command Prompt window, to pick up any changes we made to your path environment variable.
2. In the command prompt, run this. (If you did not add the pyviyatools path to your path environment variable, you will need to specify the full path to loginviauthinfo.py and any other pyviyatools you run, e.g. C:\ViyaHome\pyviyatools\loginviauthinfo.py😞

set current_namespace=gelcorp
set SAS_CLI_PROFILE=%current_namespace%
set SSL_CERT_FILE=%USERPROFILE%\.certs\%current_namespace%_trustedcerts.pem
set REQUESTS_CA_BUNDLE=%SSL_CERT_FILE%
loginviauthinfo.py

Expected result (showing your profile name instead of ‘gelcorp’, your ingress host instead of ‘gelcorp.rext03-0175.race.sas.com’ and your SAS Viya user – ideally an administrator – instead of ‘geladm’):

C:\Users\student>loginviauthinfo.py
Logging in with profile:  gelcorp
Getting Credentials for: gelcorp.rext03-0175.race.sas.com
Logging in as user: geladm
Enter credentials for https://gelcorp.rext03-0175.race.sas.com:
Login succeeded. Token saved.

C:\Users\student>

Pyviyatools Examples

The results of each of the following examples is from one of the GEL workshop environments. Obviously the caslibs, members of SAS Administrators group, folder ids, launcher context details and so on will vary between SAS Viya deployments.

List caslibs in your default CAS server

 

1. In the command prompt, run this (prefixed with C:\ViyaHome\pyviyatools\ or the path you to your copy of the pyviyatools, if you did not add the pyviyatools directory to your path):

listcaslibs.py

Expected output (caslibs shown depend on the caslibs visible to your authenticated user in your SAS Viya deployment – caslibs visible to my administrative user, geladm, in my SAS Viya deployment are shown in the output below):

C:\Users\student>listcaslibs.py
server,caslib
cas-shared-default,CASUSER(geladm)
cas-shared-default,Financial Data
cas-shared-default,Formats
cas-shared-default,hrdl
cas-shared-default,ModelPerformanceData
cas-shared-default,Models
cas-shared-default,Public
cas-shared-default,salesdl
cas-shared-default,Samples
cas-shared-default,SystemData

C:\Users\student>

List members of the SAS Administrators group

 

1. In the command prompt, run this:

listgroupsandmembers.py --id SASAdministrators

Expected output:

groupid,groupname,grouptype,groupproviderid,memberid,membername,membertype,memberproviderid
SASAdministrators,SAS Administrators,group,local,geladm,"geladm",user,ldap
SASAdministrators,SAS Administrators,group,local,GELCorpSystemAdmins,"GELCorp System Admins",group,ldap

Get the folderid of the /Public folder

 

1. In the command prompt, run this:

getfolderid.py --f /Public

Expected output:

creationTimeStamp ,createdBy ,modifiedTimeStamp ,modifiedBy ,version ,id ,name ,description ,type ,memberCount ,properties ,
"2023-07-13T08:08:51.478982Z","anonymous","2023-07-13T08:08:51.478982Z","anonymous","1","fef7a224-d3fd-4c03-8a51-fdd05b0cbfe0","Public","Public folder for general access.","folder","0","{'allowMove': 'false', 'uuid': '935842da-9ba0-479c-99e3-6a4e489322fc'}",

1. Compare this output with that from the sas-viya folders show command:

sas-viya --output text folders show --path /Public

Explain the access controls on the /Public folder

 

1. In the command prompt, run this:

explainaccess.py -f /Public --header

Expected output:

principal,read,update,delete,secure,add,remove,read(convey),update(convey),delete(convey),secure(convey),add(convey),remove(convey)
authenticatedUsers,grant*,prohibit,prohibit,prohibit,grant*,grant*,grant*,grant*,grant*,prohibit,grant*,grant*
SASAdministrators,grant,grant,grant,grant,grant,grant,grant,grant,grant,prohibit,grant,grant

Validate your SAS Viya platform is working

 

1. In the command prompt, run this:

validateviya.py

Expected output:

Data Collection Test Started: Logged in User
Data Collection Test Started: List Users
Data Collection Test Started: List Base Folders
Data Collection Test Started: List CAS Servers
Data Collection Test Started: List CAS Server Metrics
Data Collection Test Started: List CAS Server Caslibs
Data Collection Test Started: List CASLib Tables
Computation Test Started: Run Test SAS Code
name ,id
"geladm","geladm"
name ,id
"SAS LDAP Service Account","sasldap"
"SAS System Account","sas"
"CAS System Account","cas"
"SAS Administrator","sasadm"
"SAS Test User 1","sastest1"
"SAS Test User 2","sastest2"
"geladm","geladm"
"Douglas","Douglas"
"Delilah","Delilah"
"Alex","Alex"
"Amanda","Amanda"
"Ahmed","Ahmed"
"Fay","Fay"
"Fernanda","Fernanda"
"Fiona","Fiona"
"Frank","Frank"
"Fred","Fred"
"Hamish","Hamish"
"Hazel","Hazel"
"Heather","Heather"
"Helena","Helena"
"Henrik","Henrik"
"Hugh","Hugh"
"Santiago","Santiago"
"Sarah","Sarah"
"Sasha","Sasha"
"Sean","Sean"
"Sebastian","Sebastian"
"Shannon","Shannon"
"Sheldon","Sheldon"
"Sophia","Sophia"
"hrservice","hrservice"
"salesservice","salesservice"
"financeservice","financeservice"
name ,description
"Users","Base Folder for all user folders."
"Public","Public folder for general access."
"Products","Products folder for samples, canned objects and other SAS-provided content."
"gelcontent","Base folder"
"Conversational Flows","Conversational Flows Folder"
"Model Repositories","None"
name ,host ,port ,description
"cas-shared-default","sas-cas-server-default-client","5570","controller"
serverName ,systemNodes ,systemCores ,cpuSystemTime ,memory
"cas-shared-default","4","32","0.013651","1319040"
name ,scope ,description
"CASUSER(geladm)","global","Personal File System Caslib"
"Financial Data","global","gelcontent finance"
"Formats","global","Stores user defined formats."
"hrdl","global","gelcontent hrdl"
"ModelPerformanceData","global","Stores performance data output for the Model Management service."
"Models","global","Stores models created by Visual Analytics for use in other analytics or SAS Studio."
"Public","global","Shared and writeable caslib, accessible to all users."
"salesdl","global","gelcontent salesdl"
"Samples","global","Stores sample data, supplied by SAS."
"SystemData","global","Stores application generated data, used for general reporting."
serverName ,caslibName ,name
"cas-shared-default","systemData","AUDIT"
"cas-shared-default","systemData","AUDIT_ACTIVITIES"
"cas-shared-default","systemData","AUDIT_ACTIVITIES_PROPERTIES"
"cas-shared-default","systemData","AUDIT_ACTIVITIES_URIS"
"cas-shared-default","systemData","SASVIYATYPES"
runSuccessful ,jobState
"True","completed"

From here on, you should be able to run sas-viya commands and run any of the pyviyatools as permitted by the versions of the software you have available, and the permissions of the user you defined in your .authinfo file. The SAS Viya CLI has an extensive set of plugins, documented in SAS Help Center here:
https://go.documentation.sas.com/doc/en/sasadmincdc/default/calcli/n1vth8mtb8ipprn1prz5j26p3nvc.htm.

pyviyatools provides many convenient tools/scripts which my GEL colleague Gerry Nelson and I, and a number of other contributors have written to enable or simplify certain tasks in SAS Viya. It is documented in markdown files in the pyviyatools project in GitHub, particularly in:

• README.md: https://github.com/sassoftware/pyviyatools/blob/master/README.md
• INSTALL.md: https://github.com/sassoftware/pyviyatools/blob/master/INSTALL.md
• loginviauthinfo.md describes how to use loginviauthinfo.py, one of our most useful tools, to authenticate to SAS Viya before running either pyviyatools or SAS Viya CLI commands: https://github.com/sassoftware/pyviyatools/blob/master/loginviauthinfo.md
• EXAMPLES.md: https://github.com/sassoftware/pyviyatools/blob/master/EXAMPLES.md
• SUPPORT.md: https://github.com/sassoftware/pyviyatools/blob/master/SUPPORT.md
• CONTRIBUTING.md: https://github.com/sassoftware/pyviyatools/blob/master/CONTRIBUTING.md

All the tools I have tried in the course of writing this guide worked well on Windows, as they are supposed to. However, due to the nature of our work, my fellow pyviyatools developers and I use them in LINUX more than in Windows. The tools are supposed to work the same on both but it is possible that you may find a tool that does not work as well on Windows as it does in LINUX. Please let us know about any issues that you come across while using the pyviyatools, by raising an issue in the pyviyatools project or by emailing one of us. It would help if you mention that you are using it in Windows, if you are.

My thanks to Gerry Nelson and Allan Tham for their help in discussing and refining some of the things I included in this guide, and to Gerry in particular for his work in developing and publishing the pyviyatools.   See you next time!

Find more articles from SAS Global Enablement and Learning here.