Natural Language Processing (NLP) models, including those trained in SAS Visual Text Analytics, adhere to the same Analytics Life Cycle as other predictive & classification machine learning models. Since April of 2021, text concept, sentiment, topic, and category models could be registered with a few button clicks from SAS Visual Text Analytics into SAS Model Manager and deployed to CAS and MAS destinations.  This means that SAS text models can be managed and deployed with ease.  But, what about performance monitoring?   Why monitor model performance?   All models decay. Models become less relevant and useful over time.  Text models are no exception.  Model decay can be caused by gradual changes of the underlying data or sudden shocks and changes. Therefore, it is critical to continuously monitor the performance of models over time so that these signals can be caught early and mitigated.    As of SAS Viya 2022.09 (September 2022), you can monitor category models.  Category models leverage rules to classify documents into groups.  For example, when dealing with product complaints, a category model can try to classify which product the complaint is lodged against.   What metrics do we monitor?    Category models are a good candidate for performance monitoring since they are trained to predict a category classification. Users can either build their own category rules or use SAS Visual Text Analytics to automatically generate rules against an existing target column. Having a target column enables performance monitoring since we can determine how well our text models predicts the category.   Text category models have several similarities with predictive & classification models.  But there are fundamental differences that affect which metrics are used for monitoring.  For example, text categorization can allow for assignment of the same observation against more than one category, whereas machine learning models based on categorical targets usually assign one or the other label.    Keeping this in mind, text category models focus upon: Coverage, which is the percent of documents for which a model was able to identify a category.  Ideally, every document should have a match, so a high coverage percentage is a sign of a healthy model. Precision, which is the ratio of true positives to the sum of true positives and false positives.  This is also represented as TP / (TP + FP).  Precision tries to address the question of “How correct is my model?”.  A precision of 100% means that there are zero false matches for the model.  A low precision means that the model rules may be weak or too broad. Recall, which is the ratio of true positives to the sum of true positives and false negatives.  This is also represented as TP / (TP + FP).  Recall tries to address the question of “How many correct classifications has my model been able to capture?”.  A recall of 100% means that there are no false negatives. Overall False Positive Rate, which is the percent of times the model made a false match.  This is also represented as FP / (FP + TN).  F1 Score, which is an assessment of model quality using both Recall and Precision.  The calculation for the F1 score is F1 = 2 * (Precision * Recall) / (Precision + Recall).  Ideally, F1 should be near 100% Confusion Matrix, which shows the number of True Positives, True Negatives, False Positives, and False Negatives.   How do I create a performance monitoring report?   We can monitor the performance of category models in just a few steps:  From SAS Visual Text Analytics, assign the Category role to one column.  Performance monitoring is not supported for category models trained to predict multiple category classifications. Next, run the pipeline and register the category model. From SAS Model Manager, set the category model as champion.  When prompted, add the model output variables to the project output variables. Next, from the project Properties tab, change the Model function from Text analytics to Text categories.  Save the project. Open the Model Evaluation pane and fill out the Default training table, Actual category variable, Document Id variable, Text variable, and Predicted category variable properties.  Save the project. From the Performance tab, create a new performance monitoring definition and hit Run.   To see these steps in action, check out the following demo:     What would you like to see next for SAS Model Manager?  What feedback do you have our text performance monitoring?  Let me know in the comments or submit a request to SASWare Ballots with the Model Manager label 🗳️ ... View more

You’ve asked and we’ve responded. With the release of SAS Model Manager on SAS Viya 2023.06, we’ve added several new features based on user feedback.   Our first request came from managers and executives seeking to better understand the status of their ModelOps and MLOps projects. To better track statuses, actions, and characteristics of projects and models, we’ve made several enhancements to our tags. Tags quickly provide information about a project or model. Tags can be generated by users to organize projects by business units or automatically generated by SAS Model Manager. With our most recent release of SAS Model Manager, we’ve expanded the list of automatically generated tags as well as enabled searching and sorting by tags.  Learn more about our tagging enhancements in this demo video:     Next, we are working on adding tags into our Advanced Search so managers can quickly find projects with deployed models, where workflows are active, and which projects have performance monitoring defined. We are also working on adding tags to deployments. What other automatic tags would help you better manage Model Manager projects, models, and deployments? Let us know in the comments!   Our second request came from risk teams who wanted greater control over binning of input variables in performance monitoring. Our performance monitoring report now supports three types of bins for input variables: adaptive, quantile, and custom. The following demo video walks through leveraging each type of binning:   Want a peek of what we are working on next? Then stay tuned to our SAS Communities page. Likewise, what would you like to see in SAS Model Manager? Let me know in the comments or submit a request to SASWare Ballots with the Model Manager label 🗳️ ... View more

SAS Model Manager supports a variety of publishing destinations so users can deploy their models to the location that best fits their process needs. One such location is the SAS Micro Analytic Service (MAS). SAS Micro Analytic Service is a stateless, memory-resident, high-performance program execution service. The MAS destination is included with SAS Model Manager on SAS Viya. MAS is also included as a part of several other SAS Viya solutions, such as SAS Intelligent Decisioning.      The MAS destination hosts the model on SAS Viya, which reduces the amount of configuration work required to score new data using the model. Additionally, the MAS destination exposes a REST API endpoint to the model, making it easy to incorporate the model into a wider process or application. MAS supports execution of SAS and Python models, but users will need to configure an Python environment to run Python models.   MAS is best suited for ad-hoc transactional scoring in near-real time. For example, MAS is a great fit for determining if a credit card purchase is fraudulent right after the card is swiped. MAS is also great for scoring data in an application right after the applicant clicks submit. But for data that needs to be scored in batch, MAS is not the ideal destination. For scoring data in batch, an in-database or SAS Cloud Analytic Service (CAS) destination would be better suited.   Deploying and scoring a model using MAS just takes a few steps. In this article, I will walk through the steps as well as provide some of my best-practices.   Step 1: Deploy a Model into MAS   We can deploy a SAS or Python model into MAS in just a few clicks from the SAS Model Manager user interface. First, select your model(s) and hit the Publish button. From the Publish Models dialogue, select the MAS destination. Give your published model a memorable name before clicking the Publish button. In just a few seconds, we are ready to start scoring data over REST API using our model!     Step 2: Prepare to call REST API   REST APIs are a standard for communication between applications. Given the popularity of REST APIs, users have a variety of ways to call REST APIs. REST APIs can be called using tools like Postman or Insomnia, via cURL, using the requests package in Python, and many other methods. This allows users to call REST APIS using the method that is easiest for their skill sets or best fits their process needs.   I personally prefer programming in Python, but the following examples can easily be adapted to other methods. To call APIs in Python, I need to import two packages: requests and json.    import requests import json   Regardless of the method you use to call the REST APIs, you will need three things: the URL of your SAS Viya environment, your username, and your password. The following block prompts a user to enter this information in Python:   #Package to hide passwords import getpass host = input("Viya URL: ") username = input("Username: ") password = getpass.getpass("Password: ")   Step 3: Get access token   An access token allows us to make the REST API calls to SAS Viya. We can reuse our access token, but it will expire after a set time passes. To use your token longer, you can refresh your token or increase the timeout before the token expires.   This is a POST request to the /SASLogon/oauth/token endpoint with your username and password in the body of the request.   url = f"{host}/SASLogon/oauth/token" authBody = 'grant_type=password&username=%s&password=%s' %(username, password) headersAuth={'Accept': 'application/json', 'Content-Type': 'application/x-www-form-urlencoded'} r = requests.request('POST', url, data= authBody, headers=headersAuth, auth=('sas.ec', '')) token = r.json()['access_token']   Alternatively, there are a few other approaches you can take to create your access token, as highlighted in developer advocate Joe Furbee’s blog and Simon Edward’s article.   Step 4: Gather model information   If this is your first time calling the model, then I recommend making the next two calls. These calls leverage the Micro Analytic Score API to access and leverage modules deployed into MAS. More information about this API can be found in its documentation. In my experience, these two steps can prevent unnecessary debugging later. The first call shows you everything currently deployed into MAS. This confirms that your model was successfully deployed into MAS. Additionally, if you can’t remember the name of the published model, this is also helpful for finding the published model name.   This is a GET request to the /microanalyticScore/modules/ endpoint with your authorization token within the header.   headers = {'Authorization': 'Bearer ' + token} url =f"{host}/microanalyticScore/modules/" r = requests.request('GET', url, headers = headers, verify=False) for key in r.json()['items']: print(key['name'].lower())   You can prompt the user to save the published model name in Python using the input() function just like we saved the URL and username:   name = input("Published model name: ")   The second call returns information about your published model, including the inputs it expects and the outputs it returns. Using this information, you can ensure that you are passing the right data to the model. This is a GET request to the /microanalyticScore/modules/published-model-name/steps endpoint where you replace published-model-name with the name of your published model. You can reuse the headers from the previous REST API call.   headers = {'Authorization': 'Bearer ' + token} url =f"{host}/microanalyticScore/modules/{name}/steps" r = requests.request('GET', url, headers = headers, verify=False) r.json()   Step 5: Make REST API call   And now, we are ready to pass data to our published model! Using the information about the inputs we can format our data payload. In Python, our data payload will take the following form:   payload = '{"inputs": [("name": "numericVariable", "value": n), ("name": "characterVariable", "value": "char")]}'   After your data is in the correct format, we are ready to pass it to our deployed model. This next call is a POST request to the /microanalyticScore/modules/published-model-name/steps/score endpoint where you replace published-model-name with the name of your published model. Additionally, you will need to pass your data payload in the body of the request.   headers = {'Content-Type': 'application/vnd.sas.microanalytic.module.step.input+json', 'Authorization': 'Bearer ' + token} url =f"{host}/microanalyticScore/modules/{name}/steps/score" r = requests.request('POST', url, data=payload, headers = headers, verify=False) r.json()   If you are accessing a deployed decision rather than a model, take note that the endpoint for a published decision will be /microanalyticScore/modules/published-decision-name/steps/execute and that the input variable names may end with an underscore.   Next Steps   And now you are ready to score using models deployed into MAS over REST API! In upcoming article, I’ll dive into scoring using models deployed into CAS over REST API. But what other publishing destinations do you want to learn more about? Let me know in the comments below! Additionally, if you are a MAS power-user, what tips and best practices have you found most helpful? Let us know in that comments!       ... View more

The SAS Hackathon brought together global teams to collaborate from March 15 until April 12. Teams developed innovative solutions to real-world problems using SAS® and open source on Microsoft Azure. SAS Hackathon teams have submitted their ideas and now await as a distinguished panel of judges evaluate their submissions.     While we eagerly await the results, we can review the innovative submissions from the 72 teams who finished their hack. Teams leveraged various tools from visualization, open-source, deep learning models, digital decisioning and more. Nearly half the teams who finished their hack leveraged SAS’s ModelOps capabilities. Let’s dive into a few those teams.   Identify Parkinson’s disease with an app   Team ParkinStrike used machine learning to classify a user’s probability of having Parkinson’s disease based on their cursor movements. Their models were deployed into a REST API endpoint, which they leveraged within a custom application. Using ModelOps on SAS Viya, Team ParkinStrike were able to deploy their model into the right location and form to return actionable insights in the fight for early detection of Parkinson's disease.     Team ParkinStrike's Solution     Build a cheesy future   Team Notilyze leveraged SAS Viya to optimize cheese production and reduce food waste. Using models and business rules within their robust decision flow, they can suggest control adjustments in real-time to react to changing conditions. These adjustments allow them to use less milk and ensure their products are gouda quality 🧀   Ease return-to-office   As folks return to office, solutions like those created by the ML Jokers 2.2 team become vital. This team analyzed data from office workspaces to suggest comfortable yet cost effective heating and air conditioning strategies. ML Jokers leveraged multiple types of models and decision flows to make their recommendations.   Reduce healthcare costs   The iCarenetwork team pulled together a variety of data sources into predictive and text models to help reduce hospital costs.  They built models using SAS Viya and their favorite Python packages. Using python-sasctl, they registered their models into SAS Model Manager to test and compare their models, ensuring they select the best model for their use-case, regardless of language.   iCarenetwork's Solution   Monetize IoT Data   Team Algomine built dynamic pricing models for insurers leveraging IoT data. For the Hackathon, they teamed up with FoodWay2Health, a startup that sells healthy foods through modern vending machines. FoodWay2Health collects and shares a treasure-trove of IoT data from their vending machines. These models were combined with business rules to deliver personalized pricing for insuring the vending machines. Team Algomine’s solution was designed to improve over time, with the models automatically retraining based on new claims data.   React to Catastrophic Events in Minutes   Linktera4Insurance developed an end-to-end solution to alert insurance companies of their risk exposure minutes after a cataphoric event occurs. Every 10 minutes, the system pulls data from trusted news sources for extreme weather events. Using a decision flow deployed into SAS Container Runtime, they send email alerts to insurance providers listing the number of policies affected and the potential payout amounts, allowing insurance companies to react in minutes.   Linktera4Insurance's Solution   Keep equipment running   Using SAS, the REDE team built an end-to-end solution for mining equipment maintenance.  Operational data was collected every 60 second, which the solution used to recommend actions to keep the equipment running. The REDE team built supervised learning models in SAS and Python as well as optimization models to make their recommendation based on current conditions and constraints. They found that their solution increased throughput by 1-2%, increased quality by 3-4%, and lowered power consumption by 1.5%. With just a month using SAS tools, this team put together a golden solution with real, tangible benefits! ✨   Drumroll please!   There are too many wonderful submissions to cover in just one article, so check out the Hackers Hub to read up on more of these innovative teams!   I’m already on the edge of my seat awaiting the results of this year’s Hackathon. Join us at the SAS Hackathon Award Ceremony on June 1 st at 9 AM ET on LinkedIn or YouTube to hear the track, technology and regional winners.           Then join us at SAS Explore for the announcement of the global winner. Looking for free registration to SAS Explore? Presenters enjoy free registration and call for content is open until May 31 st .           What was your favorite submission? Let me know in the comments! ... View more

SAS Explore is back – live and in person from the Aria Resort in sunny Las Vegas! SAS Explore brings together the innovators, the problems solvers, and tech trendsetters bringing the world forward. At SAS Explore, we're celebrating the power of AI and exploring the latest advancements that are pushing the boundaries of what's possible. We’ll share best practices and showcase real-world examples of how AI is transforming the way we work. Save the date now so you don’t miss this awesome event!       Call for content is now open for SAS Explore! Have you harnessed the power of SAS® software to make faster, more productive decisions? Wrangled data from multiple sources using expert data management skills? Deployed and managed SAS® Viya® environments in the cloud? Mitigated bias in your predictive models? We want to hear from you. You’ve got the knowledge to help your fellow SAS users tackle today’s challenges, stay ahead of change, and skill up for the future.   Why Take the Stage at SAS Explore?   Help out your tech peers by sharing your knowledge 👩‍🏫 Enjoy free registration for SAS Explore 2023 live in Las Vegas 😎 Get feedback and guidance on your presentation in advance and in real time 📢 Earn major bragging rights with a speaker badge (and cool swag, too!) 🎁   If your idea is chosen, you will be asked to present a 20- or 50-minute breakout session in-person during the event, and possibly virtually to provide an artifact of the session.   Share Your Expertise on the Hottest Topics in Analytics   Submission topics include ModelOps, Decisioning, Generative AI, Trustworthy AI, and much more!   Submissions are due by May 31 st , with notifications of accepted submissions sent out in late June. Submit your idea today!   Looking for ideas from last year’s SAS Explore? See all sessions from SAS Explore 2022 here. ... View more

Are you ready to get hands-on with ModelOps? We’re offering a free trial of SAS® Viya® so you can explore the capabilities. Discover how to apply it – deploy your own models and test scenarios. To get started, take a look at the guide:   Getting Started - registering and accessing the free SAS Viya trial  Exploring SAS Viya - reviewing the ModelOps interfaces on SAS Viya Building Models - training models using pipelines Deploying Models - putting models into production  Testing Models - ensuring models work as expected within production  Monitoring Model Performance - ensuring models work as expected over time Additional Challenges & Scenarios - applying and extending your knowledge   Getting Started   First, register for your SAS Viya Free Trial environment. To register for the free trial, you will need to use a commercial email account and you can sign up for the free trial once per year.  This trial environment will be available for 2 weeks, giving you plenty of time to explore. You will receive an email once the environment is ready. Use the information provided within this email to login to SAS Viya and click the “i” information icon.      Next, watch the last 2 videos. These short videos show how a Data Scientist builds modeling pipelines and then deploys a champion model into a production system.     Exploring SAS Viya    Now, let's explore the ModelOps interfaces within SAS Viya. Start by opening the applications menu.      We will be using Build Models and Manage Models interfaces within this guide.      You can explore these interfaces at your leisure, as inspired by the 2 Exploration videos.   Building Models   Next, start exploring the Customer Churn project in Build Models. From this interface, you can train model pipelines to turn data into predictions. These pipelines all start with data, which can feed data processing nodes to clean and prepare the data for modeling. Next, supervised modeling nodes train machine learning models to predict the event of interest.    ⭐ Hint: Double click a project tile to open it.     The name of the table containing the training data can be found in the right-hand pane within the Data tab. Now, explore the Type and Role columns. The Role specifies how the variable will be used within the pipeline. For example, the variable with the Target Role represents the one the model pipeline will try to predict.      Click the Pipelines tab and select a ‘node’ (box) in a pipeline to see the configured properties in the right-hand pane or right-click to open the options menu to access the results.    ⭐Hint: Use these icons:     Click the Pipeline Comparison and Insights tabs. Notice the menu options in the Pipeline Comparison tab: Register Models and Manage Models. The Register Models option will register the selected model pipelines into the Manage Models interface.      Deploying Models   From Manage Models in the Applications Menu, open the Project pane (3rd item of the left menu) and then open the Customer Churn project. This project contains a treasure-trove of metadata and information about the project, such as the variables used, model evaluation properties, project history and much more. This interface can also be used to deploy, test, and monitor models.       Explore the information and actions available on each tab.     Click a model name to open it and view the scoring code and model properties.     Next, click the Publish button and publish a model to the CAS (cas) cloud destination. Publishing a model makes it available for production scoring.      The notification reports the status of the publish action:     Testing Models    Once the model has been published successfully, a scoring publishing validation test will be available to execute the score code in the production environment. This validates that our model is working as expected and ready for production workloads. Navigate back into the project by closing the notification dialogue and clicking the name of the project near the top of the screen. On the Scoring tab, click Publishing Validation and then the model name with today’s date.     Click the folder icon to set the Input table to CUSTOMERCHURNPR_1_Q1 and run the test.     When it completes, click the icon under Results. The Log contains details of the scoring test.   Monitor Model Performance    Models become less accurate over time, in a process known as model decay. To ensure that models are continuing to work as expected, monitoring the model's performance over time is critical. Click the Performance tab to view performance reports and their explanations.   ⭐Hint: If a report appears blank, click refresh:       Click the Edit button at the top right of the screen for the settings to create the reports. Notice the Tables settings.   Go to Manage Data in the Applications Menu and search for tables starting with CustomerChurnPR to locate the performance data sets used to create the reports.     Close the window, return to Manage Models, click Home, and view the links under Stay Connected to find the latest updated news and postings on ModelOps.   Additional Challenges   Now that you are comfortable with the interface and common ModelOps tasks, try improving the existing Customer Churn project in Build Models.   Create a new model pipeline by clicking     Use the gallery of nodes to prepare data and train models or start with a pre-built pipeline template if you prefer.     Open the Pipeline Comparison tab of Model Studio and register a model into Model Manager.     From Manage Models in the Applications Menu, open the Project pane (3rd item of the left menu) and then open the Customer Churn project. Your model should now be available within the project models.    Challenge Your Skills in a New Scenario    Next, let's apply what we've learned in a new scenario: using sensor readings on truck performance metrics to determine when to schedule maintenance for trucks in the fleet.   Open the Fleet Maintenance Maintenance Prediction project in Build Models.   Adjust the pipelines to your liking and run them.   Register a champion model into the model repository.   ⭐ Hint: Open the Pipeline Comparison tab and click    From Manage Models, open the Project pane and then open the Fleet Maintenance Maintenance Prediction project. First, open the model and publish the model to the CAS or MAS location.   ⭐ Hint: Click the 'Replace item' toggle if an error occurs.    Next, head back into the project and run a scoring publishing validation test using the ‘Fleet_train’ data source within the Scoring tab.     Now, create a new performance definition to create the performance reports. From the Performance tab, create a new definition or edit an existing performance definition. Walk through the New Performance Definition wizard and fill out the required information.      For input data, use the Public library with datasets starting with fleetp as shown above.      Finally, save and run the performance definition and view the results when it finishes.     Thank you for test driving the SAS Viya Free trial environment! Now you have trained, registered, reviewed, deployed, tested, and monitored your own machine learning models. Ready to learn more? Learn all the basics of SAS Model Manager in under a hour from our recent free webinar: How Do I Use SAS Model Manager? If you are ready to be an expert, try the Managing Models in SAS® Viya® course from our excellent education team!     ... View more

Are you a data scientist, developer, student, business, or technology partner who is ready to solve a humanitarian or business problem? Do you have a great idea but lack a team or the tools needed to transform your idea to reality? Then the SAS Hackathon is for you! 👩‍💻       What makes the SAS Hackathon so awesome? The Hackathon brings together teams from around the globe to solve real problems.🗺 Teams have a month to develop their solution - not just a few days.👩‍🔧 Teams own the IP for their solution, meaning they can take what they’ve developed in the Hackathon to market. 🛍 SAS provides free tooling as well as training and enablement to help teams get started.👨‍🏫 Teams also work under the guidance of a mentor, such as your resident ModelOps and Data Science author herself.✋   Speaking of ModelOps, let’s look at a few of the teams from the 2022 Hackathon that used my favorite SAS Software, SAS Model Manager. These teams were across the banking, insurance, manufacturing, and media industries; sat in multiple countries including Turkey, Germany, Belgium, Netherlands, India, and Sweden; and consisted of both students and industry professionals. 😎   Let’s start with the Linktera4Banking team, who built a low-code digital platform for loan decisions. Their solution infused industry know-how with analytics to create a best-of-both worlds approach.       The Complex_Processors team built an analytical solution to improve the efficiency of marketing campaigns. This team used SAS Model Manager to deploy their models to a real-time REST API endpoint to help extend their solution for delivering personalized offers to their customers.     The 2022 Hackathon teams developed so many impressive solutions and I am excited to see what folks will develop in the 2023 Hackathon using SAS ModelOps! 🎉🎉🎉   Need more reasons to join? Check out this blog on ten reasons why you should join the SAS Hackathon!   Registration for the SAS Hackathon is open until February 28 th , but  you can still join our kickoff call on February 9 th to learn more! ... View more

Data scientists using Python often have their favorite programming environment and set of packages for cleaning data and developing models. These data scientists are hard at work developing machine learning models, but how do these models fit into overarching business processes and needs? Enter SAS Model Manager: a leader in MLOps. SAS Model Manager helps organizations manage, audit, deploy, monitor, validate, and search their models. And yes, this includes Python models. Data scientists using Python lean on the SASCTL package to interact with SAS Model Manager and ensure that their models are included within a wider ModelOps and MLOps strategy.     SASCTL is an open source package developed and maintained by SAS that allows data scientists to interact with SAS Model Manager using Python. Using SASCTL, these data scientists can register their models and metadata directly from their favorite programming environment! This saves data scientists and IT time and headaches. SASCTL also allows data scientists to automatically generate score code for their models (because as data scientists know, training code does not equal score code) and much more. With the v1.8 release of SASCTL, data scientists can quickly save environment requirements for their Python models as well as select which project version their model is registered to.   Save Python environment requirements   Many organizations have difficulty maintaining Python development and production environments. Often there is a mismatch between what a data scientist is using to develop their model and what is available within production. This mismatch can prevent a model from running within production. Using containers, IT can create lightweight environments that match the development environment, but how do we ensure that the container has everything it needs?   SAS Model Manager uses a JSON-formatted file stored with the model’s metadata called a Requirements file. When publishing a Python model to a container, this file specifies which packages and versions to install within that container so the model has everything it needs to run. With the latest release of SASCTL, a new function is available for generating this Requirements file. This function takes the folder of modeling assets, including the model score code and pickle file, generated by SASCTL as it’s input.   requirements_json = pzmm.JSONFiles.create_requirements_json(model_dir)   For a decision tree classifier, our requirements would look like the following:    [ { "command": "pip install sklearn", "step": "install sklearn" }, { "command": "pip install numpy==1.23.5", "step": "install numpy" }, { "command": "pip install pandas==1.5.3", "step": "install pandas" } ]   To be effective, this function needs to run in the same environment the model was developed in and may require a manual review before use to ensure that it has the correct install name and package version. Unfortunately, some Python packages use a different name for install and import, such as the popular Scikit–Learn package. We can easily edit and amend our requirement files to ensure its accuracy.   scikit_learn_install = { "command": "pip install scikit-learn==1.2.0", "step": "install scikit-learn" } requirements_json[0].update(scikit_learn_install) print(json.dumps(r with open(Path(model_dir) / "requirements.json", "w") as req_file: req_file.write(json.dumps(requirements_json, indent=4))   Our new file would look like this:  [ { "command": "pip install scikit-learn==1.2.0", "step": "install scikit-learn" }, { "command": "pip install numpy==1.23.5", "step": "install numpy" }, { "command": "pip install pandas==1.5.3", "step": "install pandas" } ]   Specify the project version upon import   Did you know you can make requests for new SASCTL features through our GitHub page? By submitting an issue with the Enhancements tag, you can let our team know what you want to see in SASCTL next! Our next enhancement came directly from a user’s request through our GitHub page.   SAS Model Manager supports multiple project versions and now when registering a model using SASCTL, data scientists can select which project version for model import. Within the pzmmImportModel() function, a data scientist can leverage the projectVersion parameter to pass the name of the project version they would like model to go. The model is registered to the latest project version by default.     I = pzmm.ImportModel() I.pzmmImportModel(model-output-path, model-name, project-name, input-df, target-df, predict-method, projectVersion='Version 2')   These updates to SASCTL gives more functionality to data scientists working in Python to help improve decisioning making through analytics!   Want to learn more?   Analyst firm, IDC, recently praised SAS Model Manager for our support of a wide variety of languages and platforms. To learn why, check out these resources!   IDC MarketScape: Worldwide Machine Learning Operations Platforms 2022 The SAS MLOps solution outperforms vendors such as DataRobot, Databricks, Dataiku, Domino Data Lab, Iguazio, according to IDC MarketScape 2022 SASCTL Examples Using SASCTL package to deploy and manage MLflow models - a step by step introduction SAS Viya & Open Source How Do I Use SAS Model Manager? Ask the Expert Webinar ... View more

Before publishing models to a production environment, organizations often have a process to validate the model’s effectiveness as well as ensure the model is in good standing. Having a robust process to validate a model prior to production helps prevent production errors, poor decision making, and even disparate outcomes due to bias. But how can organizations prevent unauthorized models from going to production and ensure that their processes are followed. In this article, I’ve detailed three ways using SAS Viya 4 to prevent unauthorized model publishing.  SAS Model Risk Management   SAS Model Risk Management (MRM) is a tool on SAS Viya for documenting and mitigating model risk. SAS Model Risk Management integrates directly with SAS Model Manager to create a seamless flow of information between the two tools. Whereas SAS Model Manager is used by data scientists and MLOps engineers to manage, deploy, and monitor their models, MRM is used by risk analysts and risk teams. These risk teams often have their own processes and tests that depend on information provided by data scientists and MLOps engineers. Not to worry, the integration between MRM and Model Manager on SAS Viya help ensure the various teams involved can efficiently work together.   Models can be linked between Model Manager and MRM. This linkage tells risk teams which models they need to audit as well as allows for information to be shared between teams. Once a model is linked within MRM, that model can not be published from Model Manager until it has the appropriate risk sign-off and approval. The demo below walks through how to link models within Model Manager and MRM as well as the publishing restriction outcome.       User Permissions   Administrators can prevent publishing for a user or a group by creating a prohibit rule for ModelPublish/models. Users can still see models within the model repository, but they will get an error when they try to publish the model. This method can be used in conjunction with a custom workflow to create a standardized flow for validation and approval. SAS Workflow Manager is included with SAS Model Manager. Workflows designed within SAS Workflow Manager represent business processes. These workflows can be designed to include manual tasks, performed by a human, as well as automated tasks, logic gates, timers, and more. The demo below walks through how to restrict publishing for a specific user but also shows how workflows can be used to publish a model after the appropriate approvals are received.     Model Locks     Instead of preventing a user or group from publishing any model, you can also lock models from being published by anyone. No one will be able to publish a model until after the lock is removed from the model. Model locking and unlocking can be performed using REST APIs, which means they can easily be pulled into a workflow. The demo below walks through a workflow where a user selects which model to lock from publishing with the lock being removed after proper approval is achieved. This workflow is a simple example and can be extended to support a robust approval process.     Want to learn more? Analyst firm IDC recently praised SAS Model Manager for our strong governance and production capabilities. To learn why, check out these resources! IDC MarketScape: Worldwide Machine Learning Operations Platforms 2022 The SAS MLOps solution outperforms vendors such as DataRobot, Databricks, Dataiku, Domino Data Lab, Iguazio, according to IDC MarketScape 2022 SAS top-ranked in three Chartis RiskTech100 categories How Do I Use SAS Model Manager? Ask the Expert Webinar Build MLOps Workflows – An improved user experience with Model Manager Easily Promote and Use Containerized Model Deployments ... View more

Watch this Ask the Expert session to learn the steps for registering models and modeling metadata from various interfaces, running scoring tests, defining performance monitoring reports, publishing models and much more.   Watch the webinar     You will learn: How to use SAS Model Manager’s common model repository to store and validate models and modeling metadata. How to quickly deploy and use a model in production. How to monitor models for decay and data for shifting distributions.   The questions from the Q&A segment held at the end of the webinar are listed below and the slides from the webinar are attached.   Q&A In model manager in Viya 3.5, I only see Models, Projects, and Tasks. Is it different in Viya 3.5 vs. Viya 4? We were using Model Manager on SAS Viya 4 in today’s demonstration. It is the latest version of SAS Model Manager, and it is the version of SAS Model Manager that is updated monthly. The home page and deployments page are only available on SAS Viya 4. But you can still use nearly all the functionality we went through today using Model Manager on SAS Viya 3.5, including model registration, model validation, model deployment, model monitoring, and completing workflow tasks.    You enter or import metadata to validate models when importing or loading the models. How can you then compare models because you don't have common performance criteria? Yes, for the comparison reports, we are trusting the data scientists to train and assess their models using the same data sets and partitions. This may call for setting a standard process in the model training or model validation processes. Workflows are a potential option for building a standard ModelOps process. If you do want to do a true side-by-side comparison of model performance over time, I do recommend looking at performance monitoring since you can ensure that performance is calculated on the same sets of data at the same time.     Recommended Resources Getting started with ModelOps A structured approach to model validation SAS Model Manager Community SAS Model Manager Documentation SAS Model Manager Homepage Moving from SAS®9 to SAS® Viya® Please see additional resources in the attached slide deck.   Want more tips? Be sure to subscribe to the Ask the Expert board to receive follow up Q&A, slides and recordings from other SAS Ask the Expert webinars.   ... View more

Watch this Ask the Expert session to learn the steps for registering models and modeling metadata from various interfaces, running scoring tests, defining performance monitoring reports, publishing models and much more.   Watch the webinar   You will learn: How to use SAS Model Manager’s common model repository to store and validate models and modeling metadata. How to quickly deploy and use a model in production. How to monitor models for decay and data for shifting distributions.   The questions from the Q&A segment held at the end of the webinar are listed below and the slides from the webinar are attached.   Q&A In model manager in Viya 3.5, I only see Models, Projects, and Tasks. Is it different in Viya 3.5 vs. Viya 4? We were using Model Manager on SAS Viya 4 in today’s demonstration. It is the latest version of SAS Model Manager, and it is the version of SAS Model Manager that is updated monthly. The home page and deployments page are only available on SAS Viya 4. But you can still use nearly all the functionality we went through today using Model Manager on SAS Viya 3.5, including model registration, model validation, model deployment, model monitoring, and completing workflow tasks.    You enter or import metadata to validate models when importing or loading the models. How can you then compare models because you don't have common performance criteria? Yes, for the comparison reports, we are trusting the data scientists to train and assess their models using the same data sets and partitions. This may call for setting a standard process in the model training or model validation processes. Workflows are a potential option for building a standard ModelOps process. If you do want to do a true side-by-side comparison of model performance over time, I do recommend looking at performance monitoring since you can ensure that performance is calculated on the same sets of data at the same time.   Recommended Resources Getting started with ModelOps A structured approach to model validation SAS Model Manager Community SAS Model Manager Documentation SAS Model Manager Homepage Moving from SAS®9 to SAS® Viya® Please see additional resources in the attached slide deck.   Want more tips? Be sure to subscribe to the Ask the Expert board to receive follow up Q&A, slides and recordings from other SAS Ask the Expert webinars.   ... View more

Hi SAS Data Science Community,   I’m presenting a live “Ask the Expert” webinar on January 5 th , 11 AM – Noon ET entitled How Do I Use SAS® Model Manager? I’ll cover the steps for registering models and modeling metadata from various interfaces, running scoring tests, defining performance monitoring reports, publishing models and much more. Bring your questions. I’ll host a live Q&A session at the end of the webinar.   You will learn: How to use SAS Model Manager’s common model repository to store and validate models and modeling metadata. How to quickly deploy and use a model in production. How to monitor models for decay and data for shifting distributions. Reserve your spot for the webinar.   Want more tips? Be sure to subscribe to the Ask the Expert board to receive follow up Q&A, slides and recordings from this and other SAS Ask the Expert webinars.   Can't join the live event? You can view this and other Ask the Expert sessions on-demand here.      ... View more

The new year is just around the corner and if you have a new year’s resolution to learn more about ModelOps, then SAS has a FREE webinar for you! Start your new year off right by learning how to use SAS Model Manager on January 5 th at 11 a.m. ET. SAS Model Manager helps organizations build a streamlined ModelOps process by connecting data scientists, MLOps engineers and business analysts. It also deploys SAS and open source models quickly in a governed and standardized manner so you can make better decisions using analytics.       Join us as we walk through the steps for registering models and modeling metadata from various interfaces, running scoring tests, defining performance monitoring reports, publishing models and much more. This webinar is for data scientists, ModelOps/MLOps engineers or IT teams across industries who are new to SAS Model Manager and want to develop a mature ModelOps/MLOps practice at their organizations.   During this webinar, you will learn: How to use SAS Model Manager’s common model repository to store and validate models and modeling metadata. How to quickly deploy and use a model in production. How to monitor models for decay and data for shifting distributions   Sophia Rowland, Data Scientist and Product Manager for SAS Model Manager, will also be available to address your questions about SAS Model Manager, ModelOps, and MLOps. You don’t want to miss this free webinar in 2023! ... View more

Last year, SAS released SAS Container Runtime, the game-changing runtime engine that publishes SAS models and decisions into containers. SAS Container Runtime executes models and decisions outside of SAS Viya using standard technologies. It is lightweight, to save on-cloud costs, portable, to enable easy movement across environments, and scalable, to meet traffic needs. IT and ModelOps Engineering teams can use SAS Container Runtime to create repeatable Continuous Integration and Continuous Delivery (CI/CD) processes to efficiently test and promote new models and decision flows. To make this process even easier, SAS is providing an example template to validate and promote a SAS container through testing and production. After initial configuration of the template and environments, a SAS container can move through the CI/CD process with just a few button clicks.   SAS Container Runtime SAS Container Runtime solves multiple problems. First, it allows modeling and decisioning code to move from the development environment to the production environment in an easy, cost-efficient, and trustworthy manner. It allows organizations to move scoring code from their development environment on SAS Viya, a data and analytics platform, to small, fast, scalable, and disposal systems on their production environment where applications can access modeling and decisioning scores over REST API.     SAS Container Runtime creates a runtime container for SAS code. A container is a standard unit of software that packages code and all its dependencies so that the application runs quickly and reliably on all environments. This means that the container can be executed in any cloud or on-premise. It also means that you container does not need SAS Viya for execution.     To summarize, there are several benefits to using containerized model and decision deployments. Containers are lightweight and have small footprint to save on-cloud and infrastructure costs. SAS Container Runtime uses standard technology to fit within preexisting CI/CD processes and DevOps skill sets. Containers are portable to make moving across environments easy. Containers can be leveraged on any cloud or on-premise. SAS Container Runtime uses non-proprietary technology and can be executed outside of SAS Viya. Containers can be scaled to better meet traffic needs and duplicated to be made highly available. And finally, creating a CI/CD process around SAS Container Runtime has never been easier!   CI/CD Promotion Template   The CI/CD promotion template provided by SAS uses a GitOps process to move the container across environments automatically, saving your IT team time and resources.    In this process, SAS Viya acts as the development environment. You will use a few resources on Azure, including Azure Container Registry, where a copy of the container is stored, and Azure Kubernetes Service, which acts as the testing and production environments. You will also use a few open-source orchestration tools, including GitHub, Flux CD, and Jenkins. GitHub will store a copy of the container files, including a deployment descriptor. This deployment descriptor provides instructions for the GitOps controller, including instructions on where to find the container within Azure Container Registry, how to communicate with the container, what namespace it should be deploying to within Azure Kubernetes Service, and how many processing nodes should be within the cluster. An open-source tool called Flux CD is used to act as the GitOps controller because it provides integration between the deployment descriptor in the GitHub repository and the Kubernetes cluster. And finally, Jenkins, a popular open-source automation tool is used.   The process starts after a model or decision has been developed on SAS Viya. Models and decisions can be published directly into Azure Container Registry just as the deployment descriptor is pushed into the GitHub repository. For model publishing, the CI/CD promotion assets include a SAS Workflow Manager definition. This workflow definition containerizes the champion model within a project, deploys the container into Azure Container Registry, pushes the deployment descriptor into the GitHub repository, and then starts the Jenkins pipeline. For users publishing decisions or non-champion models, the models and decisions can be published from SAS Viya and the Jenkins pipeline can be started manually using the published name as a parameter.     The Jenkins pipeline passes the deployment description to Flux CD, where it makes the appropriate changes within the testing Kubernetes Cluster on Azure Kubernetes Service. After the testing cluster has been configured and the container is running, Jenkins runs the data through the container to validate that it is scoring as expected. Upon successfully validating the container in the testing environment, Flux CD repeats the process on the production environment. After the container is running, it is ready to score data.   Organizations can pass data to the container using the REST API. REST API is a standard for communication between applications, so this makes it easy to leverage container scoring within an application or process.   Ready to Get Started? To start using the new CI/CD promotion pipeline template, you must first add a few configurations to each environment that the pipeline touches. After you have completed the configurations, the pipeline automates promotion and validation, which saves time and reduces future headaches.  Click here to view specific instructions about configuring the environments and using the template that are available in the Model Management Resources repository on GitHub.   More information about using the promotion template and configuring each environment can be found in this recent webinar, Best Practices for Adopting Containers within your MLOps Process.   With the release of this CI/CD template, organizations can easily take advantage of containers and SAS Container Runtime. With a few button clicks, the containerized score code is moved from the container registry to the test environment. After passing validation checks, the container is moved to production where it is ready to score data. Creating a CI/CD process around model containerization has never been easier! What would you like to see next for SAS Container Runtime? Comment your thoughts below!     Want to Learn More about SAS Container Runtime? Check Out These Articles and Sessions!  SAS Explore 2022 – General Session, Unified ModelOps and Containerized Deployment for All Models Session, & Game Changing Decision Engine: SAS Container Runtime Session Best Practices for Adopting Containers within your MLOps Process Webinar What's New with SAS Model Manager? SAS Runtime Container ModelOps and DecisionOps for the Modern World SAS® Container Runtime: Programming and Administration Guide SAS® Container Runtime GitHub ... View more

There is no denying it: ModelOps is a hot topic. Gartner recently called out model operationalization as a top trend on their hype cycle and predicts rapid growth and maturity in this area. ModelOps is an enterprise effort for improved decision-making. By adopting a ModelOps process, organizations can reduce the time it takes to put models into production through the implementation of a standardized and governed process. Gartner expects that by 2025, 70% of organizations will have ModelOps architectures due to the rapid maturity of AI orchestration initiatives. As organizations ramp up these initiatives, demand for ModelOps skillsets will increase.   Image from The 4 Trends That Prevail on the Gartner Hype Cycle for AI, 2021     To keep up with industry trends, SAS recently released a certification centered on ModelOps! As I discussed last year, there are numerous reasons why a data scientist would be interested in a professional certification. First, it is a great way to pick up a new skill or even improve an existing skill. Certifications can also help with professional and career development. I recently had the opportunity to earn SAS’s new ModelOps certifications, so in today’s article, I want to share my thoughts around the new certification and point you to resources to help you prepare!   The SAS® Certified ModelOps Specialist is a 65 question exam that covers the entire ModelOps process. The exam covers everything from defining the ModelOps framework, developing the model proposal, model development, model management, model deployment, and model productionalization.  The questions tend to focus on methodologies more than tooling, with questions that range in difficulty.   As you prepare for the exam, I suggest first starting with the free SAS Webinar: How to Prep for the SAS® Certified Specialist: ModelOps Certification. The webinar will provide an overview of the exam and address frequently asked questions. While knowledge of SAS Model Manager is not required, it is helpful. SAS Model Manager helps organizations adopt a mature ModelOps process, so by taking the Managing Models in SAS® Viya® course, you have an opportunity for hands-on ModelOps learning. To accompany the certification exam, SAS has also created a course specifically around ModelOps, titled ModelOps: Governing AI and Machine Learning Models That Drive Your Business with SAS. I highly recommend checking out this course as well. This course covers an Introduction to ModelOps, the Analytics Lifecycle, Model Risk Management, SAS Container Runtime, and Administration. In this course, you will learn how to:   Distinguish ModelOps from DevOps. Use corporate culture as a positive influence on modeling outcomes. Control the process of deploying models with the ModelOps approach. Recognize the hidden impact of model risk. Deploy models in the cloud using SAS Container Runtime. Set up administration of the SAS modeling environment.   Now you should have everything you need in your toolkit to become a ModelOps expert and take on the SAS® Certified ModelOps Specialist Certification Exam! You can next take your skills and improve outcomes at your organization through robust analytically driven decisions.   What other resources do you use to stay up-to-date on trends and best practices in ModelOps? Comment below! ... View more