With the release of SAS Model Manager 2025.06, you can now publish SAS models into Databricks on Azure and AWS with the click of a button. Behind the scenes, SAS Model Manager uses PROC ACCELERATOR to push the SAS models into the Spark Database within Databricks. Pushing the modeling logic into the database allows models to execute faster on batch data as it reduces data movement. PROC ACCELERTOR does not require CAS and offers a framework to easily add new publishing destinations in the future.   In this article, we will review how to configure Databricks for publishing, how to create the publishing destination, and how to publish the model from SAS Model Manager.   Configuring Databricks   If you’ve already configured your Databricks Deployment for PROC ACCELETATOR, then good news! You’re nearly ready to publish SAS models from SAS Model Manager to Databricks. If not, the Databricks Deployment Steps will help you get started. (Note that in 2025.03, SAS changed the redistributed JDBC drivers used by SAS/ACCESS Interface to Spark. You can review this page for resolving any issues related to JDBC drivers.)   Creating the Publishing Destination   To create the publishing destination for SAS Model Manager, leverage the models plug-in to the SAS Viya CLI. Before creating the publishing destination, you must create a profile and sign in. Finally, run the following CLI command, replacing the red options with your preferred names, user or group access, and information for your Databricks deployment. The databaseConnection argument is optional.   sas-models-cli destination createDatabricks --name nameForTheDesintation --credDomainID nameForTheDomain --identityId userOrGroupIDThatHasAccess --identityType selectUserOrGroup --databricksUserId token --databricksAccessToken databricksAccessToken --databaseHost databricksHost --databaseSchema model --databaseConnection defaultStringColumnLength=1024 --httpPath "JDBCorODBCpathForDatabricksCluster"   You can also use options like overwriteInputTable to copy the input data from CAS to the spark library and keepOutputResult to leave the output in your spark library after execution of the publishing validation tests.  After running the CLI command, you’ll be able to see Databricks as an option in the drop-down menu of SAS Model Manager for users and/or groups with access.   Publishing from SAS Model Manager   Once Databricks has been configured and the publishing destination has been created for SAS Model Manager, you just need to select your SAS models, hit the Publish button, and select the Databricks destination from the drop-down menu. Once published, you can run publishing validation inside Databricks to score data from SAS Model Manager.     The published model can also be called via SAS code using the RUNMODEL statement from PROC ACCELERATOR.   Next Steps   What publishing destination would you like to see next for SAS Model Manager? Let us know in the comments! ... View more

Model versioning is a critical capability for many of our users. Versioning allows users to make alterations to a model over time while tracking those changes. Additionally, those with robust needs for model governance have built model validation processes centered on a model version. These users had previously asked to use any version of a model in a decision flow, score test, performance monitoring report, and deployment. All of this was available by the 2024.01 release of SAS Viya.   The next request we took on was to make version bumping, when publishing or setting a champion role, optional. With the release of SAS Viya 2025.05, you can decide whether to create a new model version for these two actions. But, with this option, the use of model version locks is expanded to prevent editing published or champion model versions.   Up until this point, you could publish the latest version of a model and see that the model version went through a major version increase automatically. Or you could set a model as champion and see an automatic minor version increase. For example, you might publish version 2.0 of a model. This version would be published, and model version 3.0 would be created. Version 2.0 would be read-only, meaning that it could not be edited. This mechanism enforced an audit history for published models. Model version 2.0 in SAS Model Manager would exactly match what was published. So, if you ever wanted to see what was published in each destination, you would only need to check what was in SAS Model Manager.   Setting a model champion had a similar process. You might set a model version 1.2 as champion and see that model version 1.3 was created. Version 1.2 would be read-only, again to preserve a version history. But why is a version history of the champion important? Well, there are solutions that pull models from SAS Model Manager when they are set as champion.   All of that to say: we still need to prevent editing of any model version that is published or set as champion to preserve an audit history of model versions existing in our environments and tools. Hence, edit locking. Historically, previous (i.e., non-latest) model versions have been read-only and locked from editing. Now users can lock the latest version of a model from editing if they opt not to create a new model version when publishing or setting the champion role. When they are ready to edit the model, they can create a new major or minor version in a few clicks.     When publishing a model, users will now have a checkbox to create a new version after publishing or not. If they opt to not to create a new version, the latest version will be read-only. If users want to make edits, they need to manually create a new version. Similarly, when a user sets a model as champion, there will be a checkbox to create a new version or not. And again, if they choose not to create a new version, the latest version will be read-only, but they can create a new version when they are ready to make edits. Users who always want to create a new model version when publishing or setting the champion role can leave this box checked and can continue working as they are today. But this provides others teams the ability to control when a new model version is created.  See the following demo to learn more:     What feature would you like to see next in SAS Model Manager? Let us know in the comments! ... View more

SAS Innovate is less than two weeks away, so it is time to start building your agenda. SAS Innovate is the data and AI experience for business leaders, technical users, SAS partners, and more. This year, SAS Innovate will take place May 6 th – May 9 th in sunny Orlando, Florida. Haven’t registered yet? Not to worry! Prices for individual tickets go up after April 30 th , but group discounts are available.     Let’s dive into all things ModelOps at SAS Innovate. In this article, I’ll go over the sessions, demos, and events where you can learn how to improve outcomes through better analytically driven decisions and model management.   Participate in a SAS Hackathon Boot Camp   The SAS Hackathon provides the opportunity for teams to make their ideas a reality. But you don't have to wait until fall to start your Hackathon experience. You can take part in a complementary Hackathon Boot Camp at SAS Innovate! This event offers a fantastic opportunity to gain hands-on experience, enhance your skills, and network with SAS experts and fellow participants. It’s open to customers, partners and students from any industry. All participants receive an honorary diploma of completion and a fabulously stylish and enviable SAS Hackathon Boot Camp T-shirt.       Take a Training Session   AI is innovating at top speed. To stay ahead (or at least hang on), you need to make continual learning a priority. When it comes to AI and analytics, building new skills and learning the latest technologies are the best ways to excel in your role. Our SAS training sessions make it easy to gain this knowledge as part of your conference experience.    In the SAS Viya Overview session, you’ll learn about the SAS Viya platform, including the architecture, applications, and programming features. You’ll also see how SAS Viya take you through the AI and analytics life cycle, including SAS Model Manager.   From Idea to Production With LLMs and SAS® Viya®   In this super demo, David Weik, from Into the Viya Verse fame, will take a look at how leveraging the SAS Viya platform can accelerate the time from deciding which large language models (LLMs) to use  to running prompt engineering in a no-code and a yes-code fashion, tracking every experiment, deploying our prompt as part of a decision, and monitoring usage and cost in the production system. If you’re struggling to get your LLM systems to production, this is the session for you! Related reading: Understanding LLMOps: Navigating the waters of large language models   Mapping Made Magical: Modernizing Model Implementation   Translating data into the right format for models can be a labor-intensive process, but not to worry. In this session, see how the data scientists and engineers at SAS are working on making this process painless using SAS Data Mapper. Data Mapper uses generative AI to intelligently align disparate data sources with target models, producing executable code, mapping schemas, and documentation. Through an intuitive web interface and natural language collaboration, Data Mapper accelerates time-to-value, reduces implementation costs and transforms services into scalable software to help organizations unlock analytics faster and more efficiently.   SAS Innovate After Hours   Get ready for one of the most incredible parties SAS has ever thrown—a New York City street party in the heart of Universal Studios Florida! We've reserved a select area of the park just for us—all the fun without the lines!   Responsible AI: Building ModelOps Processes With SAS® Model Manager   Yours truly will review five key capabilities available today to support Responsible AI with SAS Model Manager, including our model card, Trustworthy AI Lifecycle Workflow, model locking, fairness monitoring, and debugging issues with score code using testing. At SAS, we want to make responsible AI irresistible, meaning building AI systems responsibly should be easy and a natural part of the process. This session will demonstrate how SAS Model Manager is working towards irresistible responsible AI.     Related reading: Monitoring Model Fairness in SAS Model Manager   “Late Night” Bites and Brews in the Hub   The Innovation Hub features 60+ booths representing technology, industries, and partners. If you haven’t had an opportunity to swing by the ModelOps booth and come chat about your model deployment woes with me (or if you prefer to chat about hiking horror stories, best vacation spots, or what you’re reading), this is the time to do it. Why’s that? Well, there are free snacks, drinks, and lots of folks passionate about AI.       Using SAS Model Manager to Deploy Models Trained in SAS Viya Workbench   SAS Viya Workbench is so hot right now, but how do you get the models trained in SAS Viya Workbench into an enterprise-grade MLOps solution? Its easier than you think. In this session, I’ll discuss the integration points of these two solutions and how users can move their best models from SAS Viya Workbench to SAS Model Manager for model management, deployment, and more.   Related reading: SAS Viya Workbench & SAS Model Manager for Model Training and Deployment: SAS Models   Face Off Against the Orlando Magic   Look out for SAS’s new interactive video game that allows you to face off against Magic guard Cole Anthony. I had a blast testing it out and learned something new about myself (that I have a wingspan that’s larger than my height) as well as iterated what I already know (with my shooting percentage, I’ll never make it in the NBA). Test yourself or compete against your teammates to see who has what it takes!   Data Ethics and AI Governance   What has SAS been Innovating on? Hear directly from SAS’s Vice President of Data Ethics, Reggie Townsend, and Senior Data Scientist, Allie DeLonay, about what SAS is working on next for AI Governance. Learn about how you can address today’s most pressing AI risks while enabling responsible innovation. Be among the first to see a unique governance solution that can be applied to your organization’s AI initiatives, regardless of technical background.   From Conversation to Decision: Building an AI Assistant With SAS® Intelligent Decisioning and Azure OpenAI   Discover how to enhance AI assistant workflows by integrating SAS Intelligent Decisioning with Azure OpenAI. This guide provides a step-by-step approach to building your own AI assistant capable of redirecting chatbot conversations into trusted internal workflows. We'll demonstrate how Azure Logic Apps connects to a decision designed with SAS Intelligent Decisioning, enabling real-time decision logic execution.   Visit the Braindate Lounge   To foster stronger connections, SAS has added Braindate to SAS Innovate! Braindates are 1:1 conversations or group sessions around a specific topic. SAS Innovate attendees can create their own Braindates or sign up to join an existing Braindate. I’ll be hosting a group session all about Model Cards. You can bring your questions, feedback, recommendations, or wish-list to this session. Spaces are limited, so check out the available Braindates in the event app!     Assessing Model Bias and Model Health in SAS® Viya®   In this hands-on workshop, learn how to make decisions from machine learning models that are fair and explainable—and evaluate the health of your models once they’re deployed. You'll learn to assess models for unintended bias with Fairness and Bias statistics in SAS Model Studio. Then you'll learn how to assess the health of a deployed model using model cards.   Related Reading: Model cards: The AI transparency label you need   Model Governance – A Deep Dive: Using Python Models, Model Cards and SAS® Viya®   Haven’t gotten enough model cards and SAS Viya Workbench? This deep-dive session focuses on the development, deployment, and documentation of Python machine learning models using SAS Viya Workbench and SAS Model Manager. Learn to build a simple Python model in SAS Viya Workbench, seamlessly register it in SAS Model Manager for governance and create a model card. We'll explore key components of a robust model card, including purpose, intended use, limitations, and ethical considerations. By the end, you'll understand the entire model life cycle within the Viya ecosystem—development, deployment, and responsible documentation.   Related reading: Creating a Model Nutrition Label: Model Cards for Python Models   Networking Meets Fitness   Just because you’re at a conference doesn’t mean you need to miss your workout. But it does mean that you can work out while meeting analytics leaders and experts. Check out many of the morning exercise classes to start the day right or join a lunchtime mindfulness break to refresh your mind before diving into more learning and world-class sessions.     Ready to Learn More?   Whew, that’s a lot of great sessions! Which one are you most excited about? Did I miss a must-see session? Let me know in the comments below! And don’t forget to register! ... View more

Last year, SAS released the Trustworthy AI Life Cycle Workflow for use with SAS® Model Manager and SAS® Workflow Manager. We designed this workflow to work within SAS Viya and enforce the National Institute of Standards and Technology’s (NIST) AI Risk Management Framework, a robust tool for facilitating more transparent and inclusive analyses of AI systems.    Since then, we’ve been working to improve the SAS Trustworthy AI Life Cycle—to make it not only more accessible but also easier to use. In collaboration with our Open Source Program Office and Data Ethics Practice, we launched a new, more flexible iteration of the tool, one that is platform agnostic and better suited to collaborative practice. By following its instructions, teams can review, document, and audit their analytics projects using any platform.   And today, we’re pleased to announce that version 1.0 of the SAS Trustworthy AI Life Cycle is now available on GitHub. In this article, I’ll dive into the purpose of the GitHub project, demonstrate how teams can get started using it, and explain how you can contribute to a more responsible AI landscape.   What is the SAS Trustworthy AI Life Cycle GitHub project?   The SAS Trustworthy AI Life Cycle outlines steps for evaluating and deploying a more trustworthy AI system. It aims to make the U.S. National Institute of Standards and Technology's (NIST) recommendations, standards, and best practices for AI risk management easier to adopt and follow.   The life cycle helps organizational stakeholders specify individual roles and expectations, gather required documentation, and outline factors for consideration. As a result, teams can produce documentation to support the assertion that the organization has done its due diligence to provide evidence that the model is fair and their processes do not cause harm.   The SAS Trustworthy AI Lifecycle splits the AI and analytics lifecycle into eight phases:   Identify Stakeholders Document Project Prepare and Assess Data Train and Assess Models Test Models Deploy Model Monitor Model Review Project To make the Life Cycle easier to use, SAS created a GitHub project that leverages GitHub Pages to transform a previously static document into a more dynamic site. Teams using the SAS Trustworthy AI Life Cycle start with the first step in section one, recording their responses to the tool’s questions and prompts and moving between steps when instructions direct them. The result is a living and auditable document that records a team’s careful thinking and planning as it works toward implementing responsible and trustworthy AI best practices.   To see the Life Cycle in action, check out this demo video:     What are the benefits of using this project?   By following the steps in this project, your team will:   Have a clear understanding of roles and responsibilities. Create an AI model leveraging best practices and aligned with recommendations from NIST. Document decisions and key pieces of information in a centralized location. Track changes made to the documentation by each stakeholder and at each time point. View document history. Have a single source of truth for each analytical project. Share or store the documentation via a link.   How can I use the Trustworthy AI Life Cycle?   Creating collaborative and versioned model documentation has never been easier! First, ensure that all of your stakeholders have an account on GitHub. Next, open the sas-trustworthy-ai-life-cycle project in GitHub. From the upper-right corner, click Fork.     In the following screen, you can rename the repository if you like. Ensure that you uncheck Copy the main branch only before clicking Create Fork.     You have now created a new repository, which will open automatically. SAS has included a workflow to automatically build and deploy the website based on the documentation. This makes the documentation easy to share with stakeholders or linked within analytical projects. To enable that workflow, click the Actions tab and then click I understand workflows, go ahead and enable them.     To help keep your dependencies secure and up-to-date, next click the Settings tab, then select Code security from the options on the left. Click Enable for Dependabot version updates.     To enable GitHub pages, click Pages while in the Settings tab. Select Deploy from a branch as the source, gh-pages and root for the branch, and click Save.     You are nearly there! You need to make a few more configuration changes, but you can also use this opportunity to customize your documentation. Click on the Code tab, then go into the Website folder, open the docusaurus.config.ts file, then click the pencil icon in the upper-right corner to edit the file. Change the following properties to match the user or profile where the repository now sits:   organizationName editUrl both instances of href   If you changed the repository name, update the following properties with the new name as well:   projectName editUrl both instances of href   You can also change the title fields to reflect your own organization or project. Once you have made the updates, click Commit changes.     Now you are ready start! To access the first step in the lifecycle, open the docs folder. The docs folder includes the eight phases of the analytics lifecycle that will walk your team from project creation through retirement. Walk through the steps in order, unless prompted to move to another step in the flow. Click the edit button to start editing the markdown files.     When you are ready to commit your answer, click Commit Changes.     Committing your changes will automatically rebuild your documentation site. You can check the progress of your site rebuild by opening the Actions tab. When complete, your will see a URL.     Opening the URL will take you to your documentation site. This site can be linked to other documentation in your analytical project, with your team’s changes automatically updating the site while being tracked in Git.       To continue the process, keep working alongside your team to complete and document the steps in the Life Cycle. Alternatively, if you prefer to clone the repository using the Git CLI, instead of forking it, you can download the sas-trustworthy-ai-life-cycle project as a zip file.      Then create a new project in GitHub and copy the string under the Quick setup.     From your terminal, create a folder for the repository and change to that directory. Next, run git clone followed by the string you just copied. Move all the files you downloaded earlier into the folder you just created.  Finally run git add . followed by git commit and git push. Now you should see the repository from GitHub with everything in it and you are ready to make the configuration changes.   How can I contribute to the open source project?   Since the project is hosted on GitHub, potential contributors can fork the project, make their suggested changes, and add those changes to a pull request. GitHub users can also raise issues or suggest enhancements using the issues tab. Project maintainers welcome and encourage your suggestions and contributions.   How can I learn more about Responsible AI at SAS?   To learn more about Responsible AI at SAS, check out the following links:   Enforce Responsible AI Best Practices: Trustworthy AI Life Cycle Workflow Available Embedding responsible AI best practices within the model lifecycle Responsible Innovation ... View more

With the new bias and fairness metrics in the Model Card on SAS Model Manager, it was only a matter of time before bias and fairness charts were added to the performance monitoring report. As of 2025.03, the wait is over! Users can now see how a model’s performance on fairness and bias changes over time. With this additional information, users can now include measures of model bias when making their decision to refresh, replace, or retrain their production models.   Measuring Model Fairness and Bias   The new fairness and bias section in the performance monitoring report displays the following metrics across each time interval: Demographic parity – the maximum pairwise difference for the proportion of observations that are binned into the event level, given the cut-off value Equal Accuracy - maximum pairwise difference across values of the selected variable for Accuracy Equalized Odds- the larger of the maximum pairwise differences across values of the selected variable for True Positive Rate and False Positive Rate Equal Opportunity- the maximum pairwise difference across values of the selected variable for Ture Positive Rate Predictive Parity- maximum pairwise difference in the predicted variable corresponding to the event level   Demographic Parity, Predictive Parity, Equal Accuracy, Equalized Odds, and Equal Opportunity are calculated using SAS’s Assess Bias Action. There is also a Group Unfairness Index (GUI), calculated using the approach outlined by Szepannek and Lübke in their paper Facing the Challenges of Developing Fair Risk Scoring Models.   The specific variables shown depend on the target type of the model. Additionally, for each value of a selected variable, the following is calculated and charted: Observation count Average prediction Various measures of model performance    This allows for quick and easy comparison of how the model may treat each group in a protected class. For example, users may find critical insights, like: If the data fed to our model under-represents specific age groups If our model gives men a higher average prediction over women If our model is more accurate on white individuals over black individuals With an understanding of the model’s current measures of bias, teams can be better informed and take appropriate action.    Adding Fairness and Bias to Performance Monitoring Reports   To add fairness and bias metrics to performance monitoring reports of classification and prediction models written in SAS, Python, or R, you must first mark variables as “Assess for Bias” in the Project’s Variable tab. The variables marked as “Assess for Bias” can be inputs to a model in the project or not used by any project models at all. The variables must be available in the data used by performance monitoring, even if the variables are not used by the model to make a prediction. While an organization may not want protected class information to influence a model’s output, that information must be available to truly validate that the model is not treating protected classes differently. Note that if a variable was marked as “Assess for Bias” in SAS Model Studio, that designation will automatically carry over to SAS Model Manager upon model registration.   Additionally, fairness and bias Key Performance Indicators (KPIs) thresholds are available in a Project’s Properties tab. These KPI thresholds are used to assess model performance in the model card for classification and prediction models.     Once variables are marked as “Assess for Bias”, create and run a new performance monitoring definition. Once completed, scroll down to see the new Fairness and Bias section.   The following demo video goes over Monitoring Model Fairness from SAS Model Studio to Performance Monitoring and the Model Card in SAS Model Manager.   Want to learn more about building Responsible AI applications? Check out the following resources: Responsible Innovation Creating a Model Nutrition Label: Model Cards for SAS Model Studio Models Creating a Model Nutrition Label: Model Cards for Python Models Assess Bias of Python Models, Automatically Create Scoring Code for TensorFlow Keras Models and More ... View more

SAS Viya Workbench is an on-demand environment for SAS and Python coding. Programmers can spin up a Viya Workbench session in the cloud in just a few seconds, configure server size to match workload requirements, and minimize cloud computing costs with self-terminating sessions. Model developers can also take the models they’ve developed on SAS Viya Workbench and register them into SAS Model Manager. From SAS Model Manager, these models can be compared, tested, deployed, and monitored side-by-side with models developed on SAS Viya and other environments. Developers can build models in their preferred tooling, like SAS Viya Workbench, and still be incorporated into wider analytical processes using SAS Model Manager.   In this article, I’ll review how to register SAS models from SAS Viya Workbench using PROC REGISTERMODEL and build a complete model card. A future article will walk through the steps for a Python model.     Build the Bridge Between SAS Viya Workbench and SAS Viya   Before you can begin moving models and modeling assets between SAS Viya Workbench and SAS Model Manager on SAS Viya, your administrator needs to configure SAS Viya to accept traffic from Workbench. For many deployments of SAS Viya, this may mean allowing traffic to SAS Viya from the Workbench IP address. You can open up a terminal in Workbench and find your Ip address by running:   curl ip.me   Once your Viya administrator has allowed traffic from Workbench, you need to generate an authorization token. By following the steps in the Authentication documentation, you can create and save a token that you can call in your code as myTokenName. If you encounter a problem generating the token, then traffic from your Workbench may not have been configured correctly. Note that tokens are only valid for one hour by default. So, if you’re working for longer than an hour, you may need to re-generate your token. Additionally, if you are seeing SSL errors when running PROC REGISTERMODEL, run this line of code ahead of the PROC:     options set=SSLREQCERT="allow";     The following demo walks through the steps required to create an access token after SAS Viya is configured to accept traffic from your workbench:     Getting Your Models to SAS Model Manager   When training models in SAS, there are two types of scoring output models may generate: Data Step and Analytical Stores. Both can be registered via PROC REGISTERMODEL, but you’ll need to select the right statement for each. When registering a Data Step model, you must specify the location of the code file via the CODE statement. Similarly, when registering an Analytical Store model, you must specify the location of the ASTOREMODEL statement.   This demo walks through a quick example of model training in SAS Viya Workbench and registration in SAS Model Manager:     Building a Model Card   Model Cards in SAS Model Manager consist of information about how well the model performed during training—and over time. To build a more robust model card, it’s best practice to separate your training data into training, testing, and validation. By splitting your data, you can calculate Generalizability, or how well your model performs on unseen data.   To complete your model card, there are some properties and metadata to fill out in SAS Model Manager, like the model usage properties and training data property.   The final piece tracks model performance over time. Within SAS Model Manager, you can create your Key Performance Indicator (KPI) thresholds, define your performance monitoring report, and then run the report.     Next Steps   The SAS Model Manager team continues to enhance model cards to compile and show more information. What would you like to see next in the Model Card? Let us know below!   Want to learn more about SAS Model Manager and SAS Viya Workbench? Check out these resources: Lights ... Camera ... Let’s See SAS® Viya® Workbench in Action! SAS® Viya® Workbench: Ask the Expert Webinar Fostering Trustworthy AI Using a Model Card Creating a Model Nutrition Label: Model Cards for SAS Model Studio Models The Ultimate Must-Have for MLOps: SAS Viya How Do I Use SAS® Model Manager? SAS IS A LEADER - IDC MarketScape: Worldwide Machine Learning Operations Platform 2024 ... View more

SAS Model Manager is a model repository where SAS, Python, and R models can be registered, tested, deployed, and monitored side-by-side. To meet rising user expectation for model execution speed, the SAS Model Manager team continues to add performance enhancements.  As of 2025.01, SAS Model Manager has released two enhancements for improving the performance and execution speeds for Python and R models. The first is an updated R base container and the second is the adoption of CAS Gateway.   CAS Gateway Adoption   SAS Model Manager has multiple execution environments, from batch execution within SAS Viya to transactional REST-API endpoints hosted on SAS Viya or outside of it via containers. The SAS Cloud Analytic Services, also known as CAS, is an execution environment that is optimized for batch. It is not just a publishing destination, but also the engine behind score testing and performance monitoring in SAS Model Manager.     Last year, the CAS Gateway action was released. This action provides a way to submit code written in Python and R in a parallel manner to increase processing speeds for large datasets. As of 2025.01, SAS Model Manager had adopted CAS Gateway in our scoring tests and performance monitoring. This has drastically decreased the time it takes to score Python and R models within SAS Viya. In our performance testing, our team is seeing a 7x improvement for scoring 1 million records in SAS Viya.   To get started using the updated score testing and performance monitoring, your administrator may need to update their configuration for External Language Access Control.  This may include updating or adding pyarrow and pandas within the Python environment.   Updated R Base Container   SAS Model Manager can build and deploy containers for SAS, Python, and R models using a technology called SAS Container Runtime. SAS Container Runtime allows models to run in any OCI-compliant system using a lightweight and scalable container that manages the dependencies required by that model. These containers are executed outside of SAS Viya without needed to pay any additional fees to SAS.    The SAS Model Manager team has already updated our Python base container to be smaller and faster. With the 2025.01 release, we have made similar improvements to our R base container. To leverage the new R base container, simply deploy a R model from SAS Model Manager on 2025.01. Now, your container will be lighter and faster!   To learn more about open-source and SAS Model Manager, check out the following resources: MLOps Uncoiled: Python’s Path on SAS Viya with SAS Model Manager Score Millions of Records in Minutes Using Python Models: Python Container Scoring Optimization MLOps for Pirates and Snakes: The Sasctl Packages for R and Python IDC MarketScape: Worldwide Machine Learning Operations Platforms 2024 Vendor Assessment ModelOps Explained: A Starter’s Guide to Deploying and Managing AI and Analytical Models SAS Viya & Open Source ... View more

Beyond the recent additions for supporting the model card, python-sasctl has also seen two new additions around scoring tests and automated score code generation. With the recent release of python-sasctl, users that leverage Python have even more functionality at their fingertips!   Create and Execute Score Tests from Python   SAS Model Manager is accessible to a variety of users, with many folks leveraging the user interface to complete tasks. But SAS Model Manager offers open APIs that many developers leverage to build on top of SAS Model Manager. This includes our python-sasctl team, who creates a Python interface to SAS Model Manager. Through this interface, Python users can already register their model, create model metadata, write model score code (we’ll talk more about this later in the article), publish models, and much more. This team continues to expand how users can interact with SAS Model Manager and in the latest release, they’ve added score testing to the list of actions available to users working solely in Python.   Within the Scoring tab of SAS Model Manager, users can select a model, a model version, and a data set, to create a scoring definition. Users can then run this definition to ensure their models are working as expected. And with the latest release of python-sasctl, users can define and execute scoring tests inside SAS Model Manager without leaving their Python notebook! Python-sasctl now includes two new functions:   ScoreDefinitions.create_score_definition() to create a scoring definition. ScoreExecution.create_score_execution() to execute a scoring definition. Running ScoreDefinitions.create_score_definition() creates a new scoring definition that is viewable within the SAS Model Manager user interface.    Running ScoreExecution.create_score_execution() kicks off the scoring definition, changing the status to running  in the SAS Model Manager user interface.    Once complete, the score test results are viewable in SAS Model Manager.    Users can now generate their score code, register their models, and test it inside SAS Model Manager ensuring that it runs as expected! But users aren’t limited to running just Python models using this method or running newly created score tests. Users can create and run scoring tests against any model in their project or just execute existing ones. For teams automating processes in Python, this opens the door to more testing functionality. To learn more about these new functions in python-sasctl, check out this notebook.   Score Code Generation Enhancements   Along the topic of score code, let’s dive deeper into what score code is and how it is used. Score code is not training code. Data scientists leverage training code to create their model using data tagged with known values for the target. The goal of training a model is to build an object that will take data as input, run calculations, and return a prediction for the target. When working in Python, this model is often saved as a pickle file. To score new data using the pickle file, someone must first load the pickle file to get the model back out. Then you send the new data to the model and see what the model predicts for the target.   Scoring new data using a model is a much easier process than training a model. And python-sasctl has been able to automatically generate scoring code for years following a simple structure: Take input data Load model Send input data through model Process and return output   But in the real-world, data can be messy and our model may be very finicky on the data that it will accept. So users will sometimes write their own score code to customize the data preprocessing going into the model, like so: Take input data Preprocess data to meet model’s expectations Load model Send input data through model Process and return output   And now instead of making these users write the whole score code themselves, the python-sasctl import model function now allows users to define what preprocessing needs to take place and pass it as a parameter to the function. This notebook provides an example for leveraging this new parameter. With the recent release of python-sasctl, users can add preprocessing to their score code generation as well as confirm that their score code works as expected within SAS Model Manager without needing to leave their Python development environment.   What would you like to see next for python-sasctl? Let us know in the comments!   To learn more about Python-sasctl and SAS Model Manager, check out the following resources: MLOps Uncoiled: Python’s Path on SAS Viya with SAS Model Manager Python-sasctl GitHub Score Millions of Records in Minutes Using Python Models: Python Container Scoring Optimization MLOps for Pirates and Snakes: The Sasctl Packages for R and Python IDC MarketScape: Worldwide Machine Learning Operations Platforms 2022 Vendor Assessment ModelOps Explained: A Starter’s Guide to Deploying and Managing AI and Analytical Models SAS Viya & Open Source   ... View more

Model Cards were added to SAS Model Manager in July 2024 and our team is continuing to add enhancements and broaden their scope. The first iteration of our Model Cards focused on prediction and classification models developed in SAS Model Studio and models developed using Python. With the release of SAS Viya 2024.11, we are now adding support to build a complete model card for SAS classification and prediction models trained in SAS Studio through updates to our Import Model macros.   The SAS Model Manager macros provide SAS programmers the ability to interact with SAS Model Manager from SAS code. SAS programmers just need to add values for the various parameters and then run the code. The Import Model Macro and the Import ASTORE Model Macro have been used to take models developed in SAS code and register them directly into SAS Model Manager, along with their various metadata files. And with our latest release, this macro now generates all the files to start building a complete model card.   Within these macros, two new parameters were added, and one existing parameter is now required. The TRAINTABLE parameter was previously optional and specifies the model’s training data. This parameter is now required and is used to calculate relative importance for the model’s input variables. The two new parameters, SENSITIVEVAR and VARIMPTABLE, are both optional. The new SENSITIVEVAR parameter is used to specify a variable to use for assessing model fairness. For each value of the sensitive variable, SAS calculates the model’s accuracy and average prediction to determine if the model is treating classes differently. This uses SAS’s assess bias action under the hood. You can read more about the assess bias action in the documentation. Since tree-based models have a built-in mechanism for determining variable importance, the VARIMPTABLE parameter is used to specify the variable importance table from a tree-base model.   To have the most robust model card, we recommend that you split your data into training, testing, and validation sets prior to model training. When leveraging the macro, specify the training percentage using the SAMPLEPCT parameter and the testing percentage via the SAMPLEPCT2 parameter. Splitting your data into training, testing, and validation sets allows more points of comparison within the model card.   Once the models are registered into SAS Model Manager, users can add additional information through their interactions with the model and project in SAS Model Manager. You can learn more about these steps in the last article on Model Cards.    To see the updated Import Model and Import ASTORE Model macros in action, check out this short demo:   Next for the model card, we are evaluating updating PROC REGISTERMODEL so SAS programmers on SAS Viya Workbench can register a model into SAS Model Manager and generate a complete model card. We are also evaluating exporting the model card as a PDF. But what would you like to see next for the SAS Model Card? Leave a comment below! ... View more

Model cards were introduced with the SAS Viya 2024.07 release and act as a nutrition label for AI models. The model cards in SAS Viya feature easy-to-understand visuals and actionable takeaways. When designing the model card, SAS made sure it contained useful and digestible information for a variety of stakeholders and would become a natural extension of the model’s lifecycle. As you develop and manage models within SAS Viya, the model card starts to populate. But what about Python models developed in environments outside of SAS Viya? Don’t worry, we didn’t forget about our Python modelers! In Part 1 of this series, we explored creating a model card for models in SAS Model Studio, and in this article, we will focus on Python models developed outside of SAS Viya. To build a complete model card, we will use the python-sasctl package and tools on SAS Viya. This example notebook includes the code you need to generate the files for the model card.       Model Card example     Getting Everything in SAS Model Manager   SAS Viya cannot automatically track actions taken outside of the platform, but data scientists training their models in their Python environments outside of SAS Viya can use several functions in python-sasctl to generate the files required for SAS Model Manager and the model card. They can also import their model into SAS Model Manager directly from their Python notebook or code. Once they are happy with their trained model, these are the functions from python-sasctl our Python-wielding data scientists will need to run: pzmm.PickleModel.pickle_trained_model() to create a binary representation (pickle) of their model. pzmm.JSONFiles.write_var_json() using their input data to write out metadata about the variables the model expects as inputs. pzmm.JSONFiles.write_var_json() using their output data to write out metadata about the variables the model generates as outputs. pzmm.JSONFiles.write_model_properties_json() to save key information about the model, including target, algorithm, description, and modeler. pzmm.JSONFiles.write_file_metadata_json() for file assignments. pzmm.JSONFiles.calculate_model_statistics() to save model performance metrics at the time of training. pzmm.JSONFiles.assess_model_bias() to examine the model’s performance and prediction across levels of a potentially sensitive variable. This step won’t apply if there are no variables available for bias assessment, but you can learn more about his function in this notebook. pzmm.JSONFiles.generate_model_card() to generate the remaining files needed by the model card. pzmm.JSONFiles.create_requirements_json() to specify the dependencies required for the Python model. This step is optional but recommended for easy dependency management. You can learn more about this function in this notebook. pzmm.ImportModel.import_model() to write the model score code and register the files you created into SAS Model Manager. Be sure to specify a project using the project parameter! Optionally, you can write your own score code, zip all the files up using pzmm.ZipModel.zip_files and import them using model_repository.import_model_from_zip.   Now, you should see your Python model in SAS Model Manager and a nearly completed model card! We'll walk through each section below and highlight any remaining tasks to complete the card.    The Model Card   The first step for making your model card is to register your model into a project. The pzmm.ImportModel.import_model() imports models into SAS Model Manager directly from a Python notebook or code file. This function has a project parameter where a user can specify which project the model should be registered. If the project doesn’t exist, python-sasctl will automatically create a project with the given name and register the model to it. Once your models are registered, you can open your models in SAS Model Manager.   The Model Card will appear as the first tab of a model instance inside SAS Model Manager when the model function is prediction or classification. You can set the model function using the target type parameter of the pzmm.JSONFiles.generate_model_card() function, but you can also update the property in SAS Model Manager directly.   Left-Hand Pane   The left-hand pane contains three sets of information: tags, the modeler, and the responsible party.  The modeler is specified using the modeler parameter of the pzmm.JSONFiles.write_model_properties_json()  function, but you can update the modeler and add custom tags to the model in the Properties tab of the model. The responsible party is the user or group responsible for the model and this field is populated at the project level. A link to the project is available near the top of the screen. Within the project, navigate to the Properties Tab and then the Model Usage section. You can update the responsible party here as well as any model usage fields.    Updating the Model Card left-hand pane   Overview   The Overview section provides an at-a-glance review of the model. It provides visuals of model health that are easy to understand and are supported by other sections of the card. This tab reports on model performance during training and over time. It also reports on influential variables, variable privacy classifications, and completeness of model card. Most of the data in the Overview section will be populated using the files created earlier,  but there are areas that will require attention when first registered.   If you are noticing a blue warning about the thresholds for your training metrics, you can review and update the thresholds for action in the project properties under model evaluation. For the performance monitoring metrics to become available, complete the steps listed in the Model Audit section of this article. To ensure the variable privacy classifications come through, scroll down to the Data Summary and complete the steps I’ve listed in the corresponding section below. If you want the “No” to change to a “Yes” in the Limitations Documented block, then complete the Model Usage section of the card, as outlined in the next section of this article. Finally, to see fairness metrics, you needed to run pzmm.JSONFiles.assess_model_bias() .   This Overview section can help provide evidence that your model is in good health or direct your attention towards areas that need attention.   Model Usage   The Model Usage section describes the intended usage, expected benefits, out-of-scope use cases, and limitations of the model. This section must be filled out manually at either the project or model level in the Properties tab. The values of the model usage are inherited from the project-level properties. However, model-specific information can be specified for each property value at the model level.    Updating the Model Card Model Usage section   Data Summary   The Data Summary section provides information about the training data from SAS Information Catalog.  When registering your model from Python, the training data property will be set automatically if you specify the train_data parameter using the pzmm.JSONFiles.generate_model_card()  function, but you may be prompted to run an analysis in the Data Summary section. If you see a button in this section, click it to run your analysis.   Once the analysis is complete, you should see a summary courtesy of SAS Information Catalog. This summary contains the number of columns, number or rows, size, status, completeness of data, information privacy classifications, data tags, and data descriptions. If there are gaps in the description, tags, and status, then these can be corrected in SAS Information Catalog. Working with data in SAS Information Catalog may require advanced permissions, so work with your data engineers or data owners to ensure your data metadata is complete. Having complete data about your data is a best practice for building trustworthy models!    Updating the Model Card Data Summary section Model Summary   The Model Summary section examines the model’s performance at the time of training, which corresponds to the training donut charts in the Overview tab. The information in this tab is populated by running the pzmm.JSONFiles.calculate_model_statistics()  function.  The Model Summary section includes information about the model target, algorithm, development tool and version, various accuracy measures across training, testing, and validation splits, generalizability, and variable importance. If you’ve also run the pzmm.JSONFiles.assess_model_bias(), fainress metrics will also appear in this tab. Overall, this tab provides a ton of information about how well the model was performing during training which can provide a baseline for monitoring model performance over time.        Complete Model Card Summary section   Model Audit   While the Model Summary section focuses on the model’s performance at the time of training, the Model Audit section reports on performance over time.  The Model Audit section also provides a deeper dive into the performance monitoring donut charts in the Overview section.   The Model Audit section relies on two capabilities of SAS Model Manager: performance monitoring and Key Performance Indicator (KPI) rules. You will set your KPI thresholds in the project properties under model evaluation. You can create a rule in just a few clicks, as outlined in the documentation or the quick demo below:   Creating KPI alert rules   Performance monitoring reviews model performance over time in batch at user-defined time points. To create a performance monitor report, you need data with the actual or ground-truth values, and SAS Model Manager graphs model performance over time. Without the ground-truth, you will only get metrics on data drift. You can create a performance monitoring report in a few moments from the project, as outlined in these steps or the demo video below:   Creating and running a performance monitoring definition    Now, you can view the latest model accuracy, fairness, and model drift metrics against your thresholds on the latest run of performance monitoring.     Creating a complete Model Card   Pulling it All Together   Even if your Python model was trained outside of SAS Viya, you can still build a complete model card! To learn more, visit the example notebook and see an example of each function called out in the article. Now you should have everything you need to create a complete nutrition label for your Python model. Post your questions, feedback, and ideas in the comments below!   ... View more

With the release of SAS Viya 2024.07, a model card is now available in SAS Model Manager. The model card in SAS Model Manager was built to be like a nutrition label for AI models. What sets the SAS Model Card apart from previous model cards is the use of descriptive visuals, to make model cards accessible to all personas involved in the analytics process, including data scientists, data engineers, MLOPs engineers, managers, executives, risk managers, business analytics, end-users, and any other stakeholder with access to the SAS Viya environment. Additionally, most of our model card populates automatically as the model is developed, managed, and deployed in SAS Viya.   Model Card example   The first release of the model card supports classification and prediction models from various sources. As users and teams develop and manage their model within SAS Viya, more of the model card will automatically populate. The model card brings together information about the training data, model performance at the time of training, and model performance over time. So, for this article, we will review how to generate a complete model card for models from SAS’s no-code / low-code interface, SAS Model Studio. But fear not! This article is just the first of a series and we will focus on Python models in part 2.   The Model Card   The first step for making your model card is to register your model. From SAS Model Studio, you can register models from the Data Mining and Machine Learning pipelines in the Pipeline Comparison tab. Select the models you want to register, then open the options menu and click “Register models”. Once your models are registered, you can open your models in SAS Model Manager.   Registering a model in SAS Model Studio   The Model Card will appear as the first tab of a model instance inside SAS Model Manager when the model function is prediction or classification. If you’ve registered your model from SAS Model Studio, ensure that you are running SAS Viya 2024.07 (or later) and that your model has the function property listed as prediction or classification.    Model Card immediately after registration   Left-Hand Pane   The left-hand pane contains three sets of information: tags, the modeler, and the responsible party.  The modeler will populate automatically for models registered from SAS Model Studio, The responsible party is the user or group responsible for the model and this field is populated at the project level. A link to the project is available near the top of the screen. Within the project, navigate to the Properties Tab and then the Model Usage section. You can update the responsible party here as well as any model usage fields.    Updating the Model Card left-hand pane Overview   The Overview section provides an at-a-glance review of the model. It provides visuals of model health that are easy to understand and are supported by other sections of the card. This section reports on model performance during training and over time. It also reports on influential variables, variable privacy classifications, and completeness of the model card. Most of the data in the Overview section will be populated automatically for models coming from Model Studio, but there are areas that will require attention when first registered.   If you are noticing a blue warning about the thresholds for your training metrics, you can review and update the thresholds for action in the project properties under model evaluation. We encourage users to update these thresholds to ensure they are appropriate for their current use case, as thresholds for action may differ from one use case to the next. For the performance monitoring metrics to become available, complete the steps listed in the Model Audit section of this article. To ensure the variable privacy classifications come through, scroll down to the Data Summary and complete the steps listed in the corresponding section below. If you want the “No” to change to a “Yes” in the Limitations Documented block, then complete the limitations in the Model Usage section of the card, as outlined in the next section of this article. Finally, to see fairness metrics, before you ran your pipelines, you needed to select a variable to Assess for Bias in Model Studio.   This Overview section can help provide evidence that your model is in good health or direct your attention towards areas that need help.   Complete Model Card Overview section   Model Usage   The Model Usage section describes the intended usage, expected benefits, out-of-scope use cases, and limitations of the model. This section must be filled out manually at either the project or model level in the Properties tab. The values of the model usage are inherited from the project-level properties. However, model-specific information can be specified for each property value at the model level.    Updating the Model Card Model Usage section Data Summary   The Data Summary section provides information about the training data from SAS Information Catalog. When registering your model from SAS Model Studio, the training data property should already be populated, but you may be prompted to run an analysis in the Data Summary section. If you see a button in this section, click it to run your analysis. Since the model card is intended to be shareable, you cannot use data stored within a personal CASUSER library.    Once the analysis is complete, you should see a summary courtesy of SAS Information Catalog. This summary contains the number of columns, number or rows, size, status, completeness of data, information privacy classifications, data tags, and data descriptions. If there are gaps in the description, tags, and status, then these can be corrected in SAS Information Catalog. Working with data in SAS Information Catalog may require advanced permissions, so work with your data engineers or data owners to ensure your data metadata is complete. Model card users can click the name of the dataset within this section and open the data within Information Catalog in a new tab and make changes quickly if they have the appropriate permissions. Having complete data about your data is a best practice for building trustworthy models!    Updating the Model Card Data Summary section Model Summary   The Model Summary section examines the model’s performance at the time of training, which corresponds to the training donut charts in the Overview tab. The information in this tab is populated automatically when a model is registered from SAS Model Studio. The Model Summary section includes information about the model target, algorithm, development tool and version, various accuracy measures across training, testing, and validation splits, generalizability, and variable importance. If you’ve selected a variable in SAS Model Studio to assess for bias, those metrics will also appear in this tab. Overall, this tab provides a ton of information about how well the model was performing during training which can provide a baseline for monitoring model performance over time.      Complete Model Card Summary section   Model Audit   While the Model Summary section focuses on the model’s performance at the time of training, the Model Audit section reports on performance over time.  The Model Audit section also provides a deeper dive into the performance monitoring donut charts in the Overview section.   The Model Audit section relies on two capabilities of SAS Model Manager: performance monitoring and Key Performance Indicator (KPI) alert rules. Performance monitoring reviews model performance over time in batch at user-defined time points. To create a performance monitoring report, you need data with the actual or ground-truth values, and SAS Model Manager graphs model performance over time. Without the ground-truth, you will only get metrics on data drift. You can create a performance monitoring report in a few moments from the project, as outlined in these steps or the demo video below:   Creating and running a performance monitoring definition    Next, you will set your KPI thresholds in the project properties under model evaluation. You can create a rule in just a few clicks, as outlined in the documentation or the quick demo below:   Creating KPI alert rules   Now, you can view the latest model accuracy, fairness, and model drift metrics against your thresholds on the latest run of performance monitoring.      Complete Model Card Model Audit section   Pulling it All Together   This demo video walks through creating a model and a complete model card:   Creating a complete Model Card   Now you should have everything you need to create a complete nutrition label for your model! Stay tuned for the next article about creating a model card for a Python model. Otherwise post your questions, feedback, and ideas in the comments below!   ... View more