SAS Visual Investigator is a highly flexible SAS platform product for intelligence analytics and full-service investigation and incident management.   For example, you can use it to:   Expose money laundering networks. Enforce regulatory compliance. Solve law enforcement cases. Defend against disease outbreaks in hospitals and assisted living facilities Find missing children and reunite them with their loved ones. Conduct legal discovery Track down and put a stop to insurance fraud and other fraud schemes.   Think anything that requires investigation or intelligence gathering.   Detect. Investigate. Analyze. Solve.     Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Overview   SAS Visual Investigator enables you to identify, investigate and govern the end-to-end life cycle of an investigation, search or inquiry. It helps you every step of the way by giving you the ability to:   Explore: Explore and prepare data Detect: Search and discover clues and insights that trigger alerts Investigate and analyze: Investigate alerts Decide: Disposition alerts Act   Step 1. Explore and prepare data   The first step to an investigation is assembling, exploring and preparing your data. Two of the countless cool things that SAS Visual Investigator brings to the table are entity resolution and fuzzy matching.     Step 2. Detect: Search & discover to detect patterns, relationships, outliers, and anything suspicious or germane   SAS Visual Investigator makes searching easy and efficient. It enables you to conduct:   Free-text searches Fuzzy searching Geospatial searches Proximity searches Field boosting (relevancy optimization)   You can also:   Refine search results using interactive filters and facets Search across all data (internal and external sources) Conduct a form search, i.e., search within a specific form field, such as surname for an individual or VIN number to find a specific vehicle Restrict searches to specific entity types, fields, comments or insights   Step 3. Analyze & investigate   SAS Visual Investigator enables you to:   Filter and visualize search results in different ways to aid investigations entities, resolve entities and relationships, and initiate actions such as adding results to investigation workspaces for further analysis For targeted investigations, easily capture views of your dynamic workspace and search visualizations (e.g., maps, timelines, grids and networks), as well as notes and images documented by the analyst Preview and refine details of search results interactively Use selection tools to highlight and collate objects for further investigation   With SAS Visual Investigator, analysts and investigators can rapidly investigate and disposition alerts and events. Or! You can eliminate information lags by automating aspects of the investigation using AI embedded within SAS Visual Investigator!  You decide where human eyes are required.     Some examples of ways that AI can be built into your investigation with SAS Visual Investigator are:   Administrators can author scenarios to automatically generate alerts for analyst review using: Rule builder Decision table SAS code Alerts may be based on: Advanced analytic models Business rules Integration with third-party systems Scores can be based on one or multiple scenarios Testing scenarios and scorecards is easy     You can:   Customize scenarios that contain the logic to detect and report unusual activity, connections Filter and visualize search results in different ways to resolve entities and relationships Initiate actions such as adding results to investigation workspaces for further analysis Use selection tools to highlight and collate objects for further investigation   Step 4. Solve: Decide and Act   Alerts can trigger actions such as:   Further analysis Interview subject Make arrest Add to do not fly list et cetera   The alerts and event management capabilities enable you to:   Prioritize and assign alerts Visualize alerts in different views to gain context Enhance alerts by adding entities and integrating and connecting data Escalate by routing alerts or changing their priorities Set up auto-disposition options Designate an alert to prompt a deeper investigation   End User Interface “Investigate and Search Data”   Gone are the days of giant bulletin boards that you have to come to the office to see.     Now have everything searchable and drillable and sharable at your fingertips from any location. AND! You can easily capture views of your dynamic digital workspace.   Here is an example of the home page, i.e., the landing page the user/analyst/investigator sees when signing in to SAS Visual Investigator.     But remember, the home page as well as all aspects of SAS Visual Investigator are completely customizable!   Here is are example of the Alerts page:       Example Timeline:     Example Networks.         Example map search.     Summary     Hopefully I have piqued your interest in SAS Visual Investigators. To learn more, see the resources listed below and keep an eye out for my next post Authoring Scenarios in SAS Visual Investigator. Scenarios hold the logic for triggering alerts and actions.   For More Information   SAS Visual Investigator courses: SAS Visual Investigator: Analysis and Investigation Building an Interface with SAS Visual Investigator SAS Visual Investigator: Your Window into SAS by Susan Trueman and Seb Charrott (12 min, 2022) Keeping Children Safe from Neglect and Abuse with SAS Visual Investigator by Steve Bennett (6 min, 2018)  SAS Visual Investigator how to videos Users What’s New in 10.3 (7 min) Performing an Advanced Search (8 min) Introducing Search Capabilities (2 min) Geospatial Search Capabilities (2 min) Using Searches, Workspaces and Insights (2.5 min) Exploring the Alert Triage Process (3 min) Exporting to Microsoft Excel (2 min) Admins What’s New in 10.3 for Administrators (11 min) Managing Groups, Users and Capabilities (4 min) Functionality and Performance (4 min)   SAS Mobile Investigator overview of capabilities; YouTube by Sebastian Charrott (9 min) “A Day in the Life of a Police Officer”  SAS Law Enforcement Intelligence – Evidence Creation     Find more articles from SAS Global Enablement and Learning here. ... View more

Tips and Tricks: Improve Forecast Accuracy Using Interactive Modeling in SAS® Visual Forecasting Q&A, Slides, and On-Demand Recording   Watch this Ask the Expert session to learn how to improve your forecasting results by adjusting model parameters for individual forecasts in the easy-to-use user interface of SAS Visual Forecasting.     Watch the webinar   You will learn how to use the Interactive Modeling node in SAS Visual Forecasting to: Tweak individual forecasts to improve accuracy. Build a model from scratch. Manually select a champion model.   The questions from the Q&A segment held at the end of the webinar are listed below and the slides from the webinar are attached.   Q&A How do you know which is the best Model? What factors should I look at? The best model is going to depend on your domain. We used MAPE, which is commonly used in economics. It's mean absolute percent error. It's very easy to interpret. That's how a lot of economists are going to determine the best model, also keeping in mind simplicity. In your field, you might also consider things like the Akaike Information criterion, Schwartz Bayesian criterion, which is going to basically reward you for a less complex model. In biological science, you might be using the root mean square error. So, this is going to be based entirely on what is your field and you could look at what people have published and what criteria they are using. If you have more detailed questions about your specific situation, you can ping me after the webinar or anytime East Coast time.   Can you see the code generated by the pipeline? You can download the code for the model, but I can't quickly show that because it is a little bit involved.  See my SAS Communities blog at https://communities.sas.com/t5/SAS-Communities-Library/Downloading-Models-from-SAS-Visual-Forecasting/ta-p/831599.   How do we know if it is overfitting or not? I'm a huge fan of a holdout sample and, frankly, I do it outside of even SAS Visual Forecasting. I like to hold out data completely outside of this process and then try the model I came up with and see if it will generalize to the new data. You can only do that if you have a long enough time period, right? We don't always have that luxury, but in this case, as you can see here, I have 13 years, which is a pretty long time period.       I could hold out the last three years and then even add a holdout within the software. It's kind of comparable to predictive modeling with visual machine learning. You can have a test as well as a validation data set. That’s what I would do if you had the luxury of a long time period.  If you don't have the luxury, you could sub sample, try that sub sample set out, and then make sure that it will generalize to the data you pulled out. You could subsample over the whole time period.   Below is a screen shot of how to set the holdout within the auto-forecasting node.      Why doesn’t SAS VF always give me the best results by default? First let me say that SAS Visual Forecasting is a huge time saver and will give you many excellent and highly accurate models automatically if you have decent time series data for a reasonable length of time.  But not every series in your data set might get a great result automatically.  I like to use the analogy of when you're having a served dinner buffet at a wedding or a conference and you have three choices - fish, crab cakes or beef tenderloin. They also get to choose their dessert, right? But then you get my cousin Eva, and she's vegan and gluten free. She's going to need a special meal, right? So just like that, some of your data sets are not going to be able to get a good forecast easily. There's something kludgy about them sometimes. You want to put eyes on them, you want to look at the graph, you want to see if maybe there were some errors. One of the things that we commonly dealt with when I was looking at Chesapeake Bay data would be lab changes. We would see a sudden drop or a sudden increase in, for example, nitrates. But it was an artifact of the data. Anytime your automatic forecasting isn't giving a good forecast, you want to dive in with your own eyes and see what's going on. Because it could be some problem with the data, but it could be that you could get a slightly better model if you tweak the model around. I consider these the problem children models that they just don't easily by default give you a good model. And in some cases, actually you're getting the best model that you can get with that data. That's what I tried to show with the petroleum Louisiana naive model, which is a random walk. It might be the best you're going to get with that data set.   Is there a way to apply customization to the model at a higher level of hierarchy? For instance, to all "coal." Not currently in the Interactive Modeling node.  The Interactive Modeling node lets you work with individual time series only.  But this is on the road map, and we hope to see it soon.   How do you select the historical data for forecasting? I'm not sure what you mean.  You want the longest time period that you have. I would always start with the longest time period that you have and would only truncate that if, when you look at it, there's big changes. Depending on the field you're in, it may be more and more volatile. If we're talking stock market prices, you may want to truncate it. But you're going to look at the whole time period first and then you may decide you're going to truncate it, if more recent is really more relevant. If you're looking at something like water temperatures, you're going to probably want to use the whole time period. There could be a long-term trend in there that you want to capture. So, it's going to depend on your data, but I would always start with the full time before I started truncating.   Is there a way to choose the domain or field in SAS to get the best forecasting models? No, that's why, for SAS Visual Forecasting, you need someone who is knowledgeable. You don't need 30,000 statisticians or 20 statisticians. But you need one, at least, forecasting expert to use SAS Visual Forecasting so they would then know the field. They would then know the domain. If you want to go back to an even simpler tool, you could use SAS Visual Analytics. It is for anybody. You don't need anyone with any expertise to use SAS Visual Analytics. That will by default give you the best and would maybe lead you into hiring a forecaster and using SAS Visual Forecasting. It won't default for your domain. You would need someone on your team who knows that.   I missed the beginning of the webinar, are there any "packages/modules," for lack of a better term, that are being used beyond Viya and visual analytics that need to be purchased to do these forecasts? Everything I showed you is part of SAS Visual Forecasting. You don't have to buy anything extra on top of SAS Visual Forecasting. And this is a Model Studio interface, which is part of SAS Visual Forecasting. So, you get the programming, the Model Studio interface, you get all of that.    Is there a sample size limit? If you are asking about a) the data size limit, SAS Viya software products are designed to work with extremely large data sets and run the analytics efficiently in parallel, but you will also need to consider a practical/feasible limit based on your compute resources and configuration.  If you are instead asking about b) how much data to include in your holdout sample or out of sample region, this would depend on how long your time series data set is, and whether you have seasonal or other cycles in the data.  For example, in my data set we had seasonal data by month for 13 years.  You could set the holdout region by setting an integer (for example, 12 or 24 months) or by setting a percentage (for example, 10%). With cyclic or seasonal data, you would want the holdout data to encompass at least one full rotation of the seasons.  For example, if you have monthly data that is seasonal, you will want to holdout at least one year (12 months).  See this link for more information on holdout samples and out-of-sample region in SAS Visual Forecasting.  https://go.documentation.sas.com/doc/en/vfcdc/default/vfug/n0mf6wi57q8huin1mroj229k4tur.htm#n09h96b3jv4cecn13vmkibwyx6ro   Is there an ARCH or GARCH option in automatic forecasting? Although SAS Visual Forecasting auto-forecasting node can consider many types of models (exponential smoothing models, ARIMA and ARIMAX models, intermittent demand models, unobserved component models) it does not consider ARCH or GARCH models.   How would you add an ARCH or GARCH model to the pipeline? ARCH and GARCH models are supported via SAS/ETS procedures; SAS/ETS is included with SAS Visual Forecasting.  These procedures require coding.  PROC AUTOREG and PROC VARMAX are the two SAS/ETS procedures you can use to create ARCH and GARCH models.  It may be possible to add these via a SAS code node into the pipeline in the future.  I will continue to investigate this.     Recommended Resources SAS Demo: Use SAS Visual Forecasting’s Interactive Modeling Node to Hone in On Accuracy by Beth Ebersole Interactive Modeling in SAS Visual Forecasting by Joe Katz Please see additional resources in the attached slide deck.   Want more tips? Be sure to subscribe to the Ask the Expert board to receive follow up Q&A, slides and recordings from other SAS Ask the Expert webinars. ... View more

AI black box models can be highly accurate, but generally lack interpretability. This can be a show stopper in regulated industries such as banking, insurance, health care and others. The lack of interpretability may even prohibit you from using the best models. Enter SAS Viya stage left. SAS Viya provides a number of inordinately helpful interpretability techniques, allowing you to get the most accurate results from complex AI models, while also enabling you to interpret your results.   Out-of-the-box interpretability techniques provided with SAS Viya include variable importance plots and rankings, partial dependency plots, local interpretable model-agnostic explanations (LIME), individual conditional expectation plots, and Shapley values. This post will focus on the Shapley values. For information on other interpretability models available out-of-the-box in SAS Model Studio, see my earlier post.   Overview   SHAP (SHapley Additive exPlanation) is a useful machine learning interpretation technique developed in game theory. (SHAP also wins my award for world’s most contrived acronym...I mean really...quite a stretch to get that P in there). The mathematician and game theorist Lloyd Shapley introduced the valuable concept of Shapley values in the 1950s.   Shapley methods provide local interpretability. Recall from my earlier post that global interpretability methods explain results for all of the data, whereas local interpretability techniques explain machine learning results for individual observations (or sometimes groups of observations).  Shapley values let you learn how much each input contributes to the model’s prediction for an individual instance. The individual instance may be a loan applicant, a credit card interaction, a patient, a web site interaction, et cetera.   Unlike some of the interpretability methods I’ve described in other blogs (LIME, I'm talking to you), Shapley values are not based directly on a local regression model. Instead, Shapley methods calculate input contributions by averaging across all permutations of the inputs in the model. This helps address potential bias associated with collinearity among inputs.   Calculating exact Shapley values can eat up a lot of compute and memory resources. Because of this, a number of methods for computing approximations to Shapley values have come into vogue. Two of these approximation methods are offered via SAS Viya coding:   Kernel SHAP (available with the linearExplainer action) HyperSHAP (available with the shapleyExplainer action)   Both the linearExplainer action and the shapleyExplainer action are in the explainModel action set.   If you prefer to use the user-friendly GUI SAS Model Studio, Hyper SHAP is the method that's available to you in the current version (SAS Viya 4). If you are using the older version of SAS Model Studio on SAS Viya 3.5, Kernel SHAP is the method available on that version.   A third method, TreeSHAP, is also now available via SAS Viya coding for random forest models and gradient boosting models only. TreeSHAP calculates exact Shapley values. It is not available in SAS Model Studio at this time.     Kernel SHAP   Calculating exact Shapley values is too computationally exhausting to be feasible for most models. The Kernel SHAP method is an approximation technique that is considerably less computationally taxing.   The Kernel SHAP method implementation in SAS Viya is based on the steps presented in Lundberg and Lee (2017). Specifically, the SAS Viya action linearExplainer with preset = “KERNELSHAP” uses the Kernel SHAP method to estimate Shapley values as follows:   Gathers variable statistics and distribution from the background dataset by modeling each variable separately Generates synthetic data based on the distributions of the input data set Weights the synthetic data depending on how close they are to the original data; specifically, changes all inputs into binary, with a value of one if the synthetic data row is close to the observation of interest, and zero otherwise Scores the synthetic data using the machine learning model you want to explain (e.g., gradient boosting, neural network) Runs weighted linear regressions on the synthetic data Calculates the coefficients. Calculates the baseline/intercept using weighted linear regression that minimizes the weighted squared residual with all the new generated data   See the SAS Documentation for an example.   HyperSHAP   The HyperSHAP method—a SAS-proprietary Shapley value approximation method—is implemented by the shapleyExplainer action. The good news is that HyperSHAP is more accurate than Kernel SHAP. The bad news is that it can be more greedy for compute power.   HyperSHAP is an approximation method that estimates the conditional expectation values without fitting a regression model. HyperSHAP is more efficient than the Kernel SHAP method because it computes expected differences for only some of the input subset combinations rather than for every combination. And although the intercept value that it calculates may not be the exact average of the original data set, it will likely be closer than the Kernel SHAP method gives you. Another advantage of the HyperSHAP algorithm is that it adjusts the intercept so that all Shapley values and the intercept add up to the final prediction of the query!   The HyperSHAP method is available either via coding or else via the current version of the SAS Model Studio interface. Coders will use the shapleyExplainer action to estimate Shapley values using HyperSHAP. Coders have the added flexibility to adjust the tradeoff between accuracy and computer intensity by adjusting the hyperparameter depth. The hyperparameter depth controls what subsets may be used to approximate Shapley values. Generally you will start out by setting depth = 1. Then try depth = 2, and so on. As you increase the depth hyperparameter you will improve the level of approximation and the more variable interaction information will likely be captured.  On the down side, you will increase the computation time and memory use needed to run the algorithm.   If you prefer a user-friendly GUI to coding, you can use SAS Model Studio.  In SAS Model Studio on Viya 4, HyperSHAP is the method used behind the scenes, but you can't change the depth. Some users have expressed that they are interested in knowing the intercept value. To determine the intercept, simply subtract all the Shapley values from the actual prediction for the instance to get the intercept. Recall that HyperShap adjusts the intercept so that all the Shapley values and the intercept add up to the predicted value!   When computing the HyperSHAP values, SAS Viya:   Enables coders to specify the depth of the approximation Generates a copy of the training data set where the inputs are set equal to the inputs for the instance under consideration for some of the subsets of the inputs Scores the new observations Averages the prediction for each data set copy Computes a weighted aggregation of the average predictions   TreeSHAP   Like the other Shapley values, TreeSHAP values enable you to understand the contribution of each input to the prediction for an individual instance. The TreeSHAP algorithm was introduced by Lundberg et al. (2020). This method is less computer intensive than Kernel SHAP or HyperSHAP, but unlike those methods it is not model-agnostic. As the name implies, TreeSHAP works only for tree-based models.   Tree SHAP computes exact Shapley values. Tree SHAP can also be used to extend local interpretations to capture input interactions and interpret global model structure based on local interpretations.   Coders can calculate TreeSHAP values using the current version of SAS Viya for gradient boosting or random forest models with at least two trees and a maximum of two splits per node. This capability became available in Stable version 2023.04 (and LTS 2023.09). TreeSHAP values can be calculated using PROC ASTORE by setting TREESHAP = 1 in the SETOPTION statement. The DESCRIBE statement enables you to see the input and target that each generated TreeSHAP value explains.   Tree SHAP modifies the Shapley computational algorithm that tracks the number of subsets that flow into each node of a tree. A big advantage of the Tree SHAP method is that it reduces the computational complexity of the calculations.   Coding Examples for Shapley Methods   A number of coding examples are provided in the SAS Viya Platform Programming Documentation     Using the GUI SAS Model Studio to Get Shapley Values (LTS 2022.09)   In the current version of SAS Model Studio, Hyper SHAP is the method available to you. Once you have built your pipeline in SAS Model Studio, select a model node (e.g., the gradient boosting node).     Open the options pane. Scroll down to Post-training Properties and expand Model Interpretability, Local Interpretability. Select HyperSHAP as shown below.     By default SAS Model Studio will select five local instances (individual observations) at random. Rerun the gradient boosting node and open the results. Under the Summary tab, you will now see a Model Interpretability button. Select the Model Interpretability button and select any of the five instances to see the Shapley values graphed.     In the graph above we are predicting cholesterol levels from weight, smoking, sex, systolic and diastolic blood pressure, and metropolitan relative weight. We see that for Local Instance 1001441, weight increased the prediction for cholesterol by more than 1 unit. Systolic reduced the prediction for cholesterol by 4 units.     If you want to select specific observations, you may specify (type in) up to five at a time as shown below.     For More Information   SAS Resources   Güneş, Funda, Ricky Tharrington, Ralph Abbey, and Xin Hunt. 2020. How to Explain Your Black-Box Models in SAS Viya SAS Documentation: explainModel details explainModel Kernal SHAP example explainModel HyperSHAP example astore syntax Xin Hunt’s response to Intuitive Way to Interpret Intercept Value of Shapley Values Output   Original Papers   Lundberg, Scott and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. Lundberg, Scott, et al. 2020. From local explanations to global understanding with explainable AI for trees Shapley, Lloyd S. 1952. A Value for n-Person Games Strumbelj, Erik and Igor Kononenko. 2010. An Efficient Explanation of Individual Classifications using Game Theory   Shapley Values Illustrated   SHAP Part 3: Tree SHAP ... View more

This blog will do a deeper dive on the mitigateBias action and how to tune it. The SAS mitigateBias uses a highly versatile approach (exponential gradient reduction) that:   Applies to any classifier family Allows many definitions of fairness, including demographic parity and equalized odds   The approach is based on A Reductions Approach to Fair Classification by Agarwal et al. 2018 and results in a randomized classifier with the minimum empirical error subject to the fairness constraints selected by the user. For a more general introduction to assessing and mitigating bias using the assessBias and mitigateBias actions of the SAS Fair AI tool set, see my previous blog Mitigating Bias Using SAS Fair AI Tools.   Exponential Gradient Reduction Algorithm   The brains of the approach is the Exponential Gradient Reduction Algorithm (EGR), which is model agnostic. That means you can use it whether your model is gradient boosting, random forest, logistic regression, neural network, decision tree, etc. EGR works by iteratively manipulating the weights for different subsections of the data. For a simple example, let’s use a binary target of BAD (1 means loan default, 0 means no default) and a binary sensitive variable of female/male for gender. Thus we have four sections:   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   By default the EGR of the mitigateBias action will manipulate these weights using the training data only. When the model is trained, each of the segments will have a different performance. The EGR algorithm works to compensate for bias by increasing the weights of segments that underperform, and decreasing the weights of segments that overperform. This is an iterative process with the learning rate and the bound working in tandem to iteratively achieve the best result.   When the weights get updated, the model compensates for the change in weights, and the performance for each segment changes. The effect of the weight depends on many things, including:   What model you are using (gradient boosting, random forest, etc.) The values of the observations in your data set in each segment The number of observations in your data set in each segment   Hyperparameter tuning the mitigateBias action in SAS Viya   If you don’t get satisfactory results with your data using the defaults, you can adjust the hyperparameters.     LogLevel   The logLevel specifies how much log information to print.  The default is currently 0 but will change to 1 in the next release (2023.11).  The logLevel meanings in the updated version will be:   0:  Suppress most notes, allow warnings and errors 1:  Default, level 0 + a few procedural and informative notes 2:  Level 1 + some more informative notes and a few notes per iteration (if you want notes per iteration this is the level to select) 3 and above:  Allows notes to print from the training program, generally used for debugging   There’s no real difference between levels 3 – 7 at this point in mitigateBias. The additional levels are reserved for future use.   Best practice is to set it at 5 for initial debugging. Once you have debugged, set it back to 1 to avoid filling up your log. Be sure to clear the log after each run.   PROTIP from Xin Hunt: If you have “print” statements in the training program and want to see them, either set logLevel to >= 3 (so all notes will be printed), or print them as a warning or error (e.g., print (warn) “some message”;). Warnings and errors are always shown regardless of logLevel.   Side note:  For mitigateBiasDecisionTree (and all future mitigateBiasXXX actions), log levels 4 and 5 print additional things, so the effective highest log level is 5.  However, this doesn’t affect mitigateBias.   How many iterations?   The default value for maxIters is ten. You can set it to the maximum of 50 if you have the time available. A high number of iterations is useful if you are working with a new data set, working on debugging, and if you need to tune some of the other hyperparameters.   In the ideal situation you will see that gradually over five to ten iterations the EGR will put the segments on an equal footing. But CAUTION! If it converges in just one iteration that may be a spurious result. This result may not be replicable with a new data set and thus will not be generalizable to the full population. On the other hand, if it takes too many iterations to converge then you may be wasting time, and you may want to increase the bound (see below).   Tolerance   Recall that there is commonly a tradeoff between accuracy and bias. Tolerance is based on how much bias the user is willing to tolerate and determines an acceptable level for stopping of the EGR. The acceptable tolerance will depend on your domain (e.g., regulated industry) and on your data. If you have highly unbalanced data, for example, you may be willing to accept a a higher tolerance in order to get results.   Tolerance is currently calculated as compared to the global average for that particular metric. For example, with predictive parity the tolerance will be calculated as the absolute value of the segment prediction minus the global average prediction. For accuracy it would be the absolute value of the segment accuracy minus the global average accuracy.   Bound, learningRate, and tuneBound   EGR is an iterative process with the learning rate (default 0.01) and the bound (default 100) working in tandem to achieve the best result over multiple iterations. There are no hard and fast rules because the effect of adjusting the bound and learning rate will depend on the model you are using (e.g., gradient boosting, logistic regression, etc.) and your data.   One best practice for tuning these hyperparameters is to initially leave the learning rate at the default, and adjust only the bound. The larger the bound, the more the weight changes on each iteration. Conversely, the smaller the bound, the less the weight changes on each iteration.   Segments with small numbers will be highly affected by weight changes. Thus if you have a high bound value, the performance/accuracy of underrepresented segments will be highly volatile (jump around a lot) leading to overshooting by the EGR. This is less of an issue with balanced data. However if the bound variable is very high, you will see this volatility even with balanced data. If bound is too low, you may see no change with each iteration, and you may never converge.   NOTE: In earlier releases of the software (LTS 2022.03 and earlier), there was an upper bound of 5,000 that could not be exceeded. Particularly in this older software, if you hit the maximum bound and still have not received satisfactory results, it may be helpful to adjust the learning rate instead. A higher learning rate may overshoot and a lower learning rate may undershoot.   If you are having trouble finding a good bound you can set TuneBound=TRUE to get a good ball park starting place to set the bound. From there you can then manually adjust to get even better results if you have time to tinker with the tuning.   TuneBound is a relatively simple meta-algorithm wrapped around EGR that sets the bound value to very high for 5 maxIter and to very low for 5 iterations. It then analyzes what the weights did in each of those 5 maxIterations. Weight changes are expected to be volatile/jumpy at the high bound. With the low bound, we would expect nonconvergence. If both situations (volatile at high and nonconvergent at low) are true, tuneBound moves the bounds inward (high bound not so high, and low bound not so low). Eventually we get results where the weights are stable at the high bound and we have convergance at the low bound. At that point, tuneBound takes the average of the two and sets the bound to that average.   Tuning will not always work   Remember that old piano that sat in a damp place for years and now the wood is warped so it can’t be tuned? Just like that piano, it’s possible that your model may not converge no matter how much tuning you do. Recognize that if you have too few observations in a segment it just won’t work. Not even with 50 iterations, not even if you play with both the bound and the learningRate. Take the extreme example of a segment of only 1 observation. Then TPR will be either 0 or 1, it can’t be anything in between. If you compare this to another segment that has 1000 observations, it will have a TPR somewhere in between 0 and 1 and your tiny segment will never match that TPR. The larger the data set and the more balanced the data (equal proportions in the different segments), the better.   Also if you have extreme bias in the data then might not be able to overcome that. For example if one segment is 100% 1s or 100% 0s in the target variable. In this case, you need to go back to the drawing board: look at your training data, evaluate the goals of the modeling effort, and so on.   Interpreting Results   Demographic (statistical) parity is achieved when the probability of a positive prediction is similar for those in different sensitive groups in the training data. With a binary target, demographic parity is satisfied for a classifier h if the prediction h(X) is statistically independent of the sensitive variable A under a distribution (X, A, Y). In other words, for a binary classifier where ŷ ∈ {0,1} if   E[h(X)|A=a] = E[h(X)] for all a.   In our simple example with gender (female/male) as the binary sensitive variable and the binary target BAD (1/0) , this means that the expected value of h(X) given gender = female should equal the expected value of h(X) given gender = male.   Here we see an example where by Iteration 3, demographic parity was low (0.001929), without much sacrifice in misclassification rate (went from 0.3330 to 0.3409).       Equalized odds. There are knows flaws with demographic parity, therefore some practitioners prefer to use equalized odds. Equalized odds match both the true positive rates AND the false positive rates for different sensitive groups. A classifier h satisfies equalized odds if h(X) is conditionally independent of the sensitive variable A given Y , i.e.,   E[h(X)|A=a, Y=y] = E[h(X)|Y=y] for all a, y.   Below, we see that we converge in 11 iterations at a tolerance of 0.18. We see a nice pattern of the equalized odds reducing but the misclassification rate not getting worse.   Thank you to my colleague Randy Collica for the data and results shown here.     Equal opportunity   Equal opportunity considers only true positive rates (TPR), and it does not take into consideration false positive rate.   PRO TIP from my colleague Xinmin Wu: Demographic parity and equal opportunity will be easier to tune than equalized odds. This is because equalized odds deals with two metrics (TPR and FPR) simultaneously, and fine-tuning them can be challenging because improving one may worsen the other. Thus the current bound-tuning logic works best for demographic parity or equal opportunity. FYI, this bound tuning feature is not available in the open-source package Fairlearn; SAS has three US patents issued pertaining to it.     FOR MORE INFORMATION   Agarwal, A., Beygelzimer, A., Dudik, M., Langford, J., and Wallach, H. (2018). “A Reductions Approach to Fair Classification.” In Proceedings of the 35th International Conference on Machine Learning, 60–69. PMLR.   SAS Documentation   ACKNOWLEDGEMENTS   Thank you to the following SAS employees who provided details, data, and questions to help provide a deeper dive into the mitigateBias action. Ricky Tharrington Xin Hunt Allie DeLonay Xinmin Wu Randy Collica   ... View more

Why Distributions are Important   Why are the distributions of your data and your errors so important? Understanding different distributions is key to understanding your data and helping you know how to treat the data and what models to use. SAS Studio lets you graph different probability distribution functions (pdfs). SAS also enables you to quickly create large data sets drawn from different distributions. This post shows you how.   Many models rely on assumptions. For example, ordinary least squares regression makes assumptions, such as:   Normality: Error terms follow a normal distribution, with mean 0 IID errors: Error terms are independent and identically distributed Homoscedasticity: Equal variance Linearity (in the parameters)    Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   If you violate the assumptions, your results will be incorrect. For example, your confidence limits will be wrong. In real life, data and errors usually stray from the normal distribution. Sometimes a large number of observations (high n) will help you out, and you will be close enough to normal that your results will be adequate and your models will still provide reliable results. However, sometimes your data and errors follow a very different distribution. For example, count data commonly follow a Poisson distribution. Data that have been drawn from two separate populations may follow a bi-modal distribution.   So depending on the distribution you discover, you may select a different model. Or you may wish to transform the data prior to modeling. Understanding the distribution is key to understand your data and selecting your best modeling options.   Common Distributions   Normal (Gaussian) Poisson Negative Binomial Gamma   Let’s see how to create pdfs and simulate data for each of these four distributions.   Normal (Gaussian)   For a normal distribution, the probability density function forms the bell-shaped curve. The mean, median, and mode are all the same for a standard normal distribution. Skewness and kurtosis would both equal zero for a standard normal distribution. Data (and errors) are rarely perfectly normally distributed, but for a large number of observations, they may approximate a normal distribution.   We can easily plot the probability density function of a normal distribution using the SAS PDF function for the normal distribution. Notice that the mean may be represented, by μ, θ, etc. depending on the source. Likewise, the variance may be represented by σ 2 or λ 2 etc.     Using the default location parameter mean equal to 0 and the default scale parameter standard deviation equal to 1, we get the graph shown below with the following code. Notice that if your standard deviation equals 1 then your variance also equals 1, because the variance is the square of the standard deviation.   /************************ * NORMAL (GAUSSIAN) PDF * *************************/ data normalpdf; do x = -3 to 3 by 0.1; y = pdf("Normal", x); output; end; run; proc sgplot data=normalpdf noautolegend; title "Normal Probability Density Function"; series x=x y=y; xaxis grid label="x"; yaxis display=(nolabel) values = (0 to .5 by .1); refline 0 / axis=y; run;     There are different ways to simulate data from specific distributions using SAS. One way is to use SAS/IML with the RANDGEN subroutine to generate random values. The second, and easier way, is to use a simple DATA step with the RAND function. This is the method we will use here. DATA step to simulated data from univariate and uncorrelated multivariate distributions RAND function to generate random values from standard univariate distributions including:   The DATA step is excellent for simulating univariate data, but SAS/IML is more powerful for simulating multivariate data.   /**************************************************** * Simulate data from NORMAL (GAUSSIAN) distribution * *****************************************************/ data normal; do i = 1 to 100000; x = rand("Normal"); output; end; run; title "Normal Distribution"; proc print data=normal (obs=10); run; proc univariate data=normal; var x; histogram; run;     In most cases , however, we do not have normality. We may have other distributions such as Poisson, negative binomial, Gamma, etc. Let’s look at these three distributions.   Poisson   The Poisson distribution models discrete non-negative numbers, usually count data. Examples include the number of fish caught by a waterman in a day, the number births per year in a city, or the number of Barbie dolls purchased per day in a store. The Poisson distribution has one parameter, the mean number of events, commonly represented as λ. Remember that the Poisson distribution is different from the exponential distribution. The exponential distribution describes waiting time between events, whereas the Poisson distribution describes the number of times the event occurs within a period.   Let’s plot the probability density function of a normal distribution using the SAS PDF function for the normal distribution.     We’ll use this code for the following graph.   /************** * POISSON PDF * ***************/ %let lambda=3; data poissonpdf; do k = 0 to 10; y = pdf("Poisson", k, &lambda); output; end; run; proc sgplot data=poissonpdf noautolegend; title "Poisson Probability Density Function for lambda = &lambda"; series x=k y=y; xaxis grid label="k"; yaxis display=(nolabel) values = (0 to .4 by .1); refline 0 / axis=y; run;     We can use the RAND function to create data from the Poisson distribution.   /****************************************** * Simulate data from POISSON distribution * ******************************************/ %let lambda=2; data Poisson; do i = 1 to 100000; x = rand ('POISSON', &lambda); output; end; run; proc print data=poisson (obs=10); title "Poisson with lambda = &lambda"; run; proc univariate data=poisson; var x; histogram; run;     By simply changing our %let macro, we can change lambda to 4 or 16 or whatever we would like. For example,   %let lambda=4;       Binomial and Negative Binomial   The binomial and negative binomial distributions are for sequential independent Bernoulli trials with a binomial outcome, e.g., success or failure. Both distributions assume that the probability of success p is the same for every trial. The binomial distribution models the number of successes in n trials. The negative binomial distribution models the number of failures before k successes occur.   The classic example is repeated coin tosses. With a fair coin, the probability of heads versus tails is 50/50. That means that p = 0.5. Each toss is independent, so each toss has a 50% chance of heads and a 50% chance of tails.   In our code, we use k instead of n to create the pdf.     /************************ * NEGATIVE BINOMIAL PDF * *************************/ %let k=5; %let p=0.5; data negbinomialpdf; do m = 0 to 20; y = pdf("NEGBINOMIAL", m, &p, &k); output; end; run; proc sgplot data=negbinomialpdf noautolegend; title "Negative Binomial Probability Density Function (m=&m, p=&p)"; series x=m y=y; xaxis grid label="m"values = (0 to 20 by 5); yaxis display=(nolabel) values = (0 to .2 by .05); refline 0 / axis=y; run;       And we can again use the RAND function to simulate data, this time from the negative binomial distribution.   /**************************************************** * Simulate data from NEGATIVE BINOMIAL distribution * *****************************************************/ %let p=0.5; %let k=10; data negbinomial; do i = 1 to 100000; x = rand ('NEGBINOMIAL', &p, &k); output; end; run; proc print data=negbinomial (obs=10); title "Negative Binomial Data with p = &p, k = &k"; run; proc univariate data=negbinomial; var x; histogram; run;     Gamma   The gamma distribution is a non-negative, continuous, two-parameter probability distribution. The pdf of the gamma distribution using shape and scale parameters is:     The gamma function is:     /************************ * GAMMA PDF * *************************/ %let a=3; %let lambda=1; data gammapdf; do x = 0 to 15 by 0.01; y = pdf("GAMMA", x, &a, &lambda); output; end; run; proc sgplot data=gammapdf noautolegend; title "Gamma Probability Density Function (a=&a, lambda=&lambda)"; series x=x y=y; xaxis grid label="k"values = (0 to 15 by 5); yaxis display=(nolabel) values = (0 to .4 by .1); refline 0 / axis=y; run;     /****************************************** * Simulate data from GAMMA distribution * ******************************************/ %let a=2; %let lambda=2; data Gamma; do i = 1 to 100000; x = rand ('GAMMA', &a, &lambda); output; end; run; proc print data=poisson (obs=10); title "Gamma Data with a = &a, lambda = &lambda"; run; proc univariate data=gamma; var x; histogram; run;       For More Information RAND function SAS documentation PDFs for Distributions SAS documentation Overlay density estimates on a plot blog by Rick Wicklin   Find more articles from SAS Global Enablement and Learning here. ... View more

The procedure PROC GAMMOD and the Generalized Additive Model (GAM) node in SAS Model Studio build generalized additive models. GAMs are able to fit non-normal, non-linear models.  This makes them quite useful!  This post will show you how easy they are to use in SAS Model Studio.   Example Use Case:   Number of births per area Predict number of births to determine need for hospitals, pediatricians, teachers, etc. Inputs can include population totals, proportion female, proportion child-bearing age, income levels, education levels, access to birth control, etc. GAMs versus GLMs versus Linear Regression   Recall the assumptions of ordinary least squares simple linear regression models: Linearity Normality IID Homoscedasticity Linearity: The relationship between the outcome and the features is linear in the parameters   From Brian Gaines https://blogs.sas.com/content/subconsciousmusings/2022/03/24/accuracy-versus-interpretability-with-generalized-additive-models-gams-you-can-have-both Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Normality: Errors follow a normal distribution   IID errors: Error terms are independent and identically distributed. Homoscedasticity: Equal variance.      In most cases we do not have normality. We may have other distributions such as Poisson, negative binomial, Gamma, Tweedie, etc. But not to worry. In these cases we can use generalized linear models (GLMs) or generalized additive models (GAMs). Generalized Linear Models (GLMs) assume linearity, but generalized additive models (GAMs) do not.  GAMs accomplish this using spline terms to generalize linear models’ assumption of linearity.  Thus GAMs are even more generalized than GLMs!   Another perk of GAMs is that they have less bias than GLMs. However, as the number of observations increases, GAMs tend to converge more slowly than GLMs.   PROC GAMMOD versus PROC GAMPL   PROC GAMMOD (SAS Viya) is very similar to PROC GAMPL (SAS/STAT SAS 9 high performance) in features, options and results. PROC GAMMOD does provide additional functionality by supporting: BY processing Creation of an analytic store for scoring The link function also differs between the procedures. PROC GAMMOD uses a log link for both the Gamma and Inverse Gaussian distributions. PROC GAMPL uses the reciprocal for a Gamma distribution and the reciprocal square for Inverse Gaussian distributions, as shown in the table below.     PROC GAMMOD versus PROC GAM   PROC GAM (SAS/STAT SAS 9) supports Gaussian (normal), binomial, Poisson, Gamma and inverse Gaussian distributions. PROC GAMMOD (SAS Viya) supports all of these PLUS negative binomial and Tweedie distributions. In addition, PROC GAMMOD and PROC GAM work very differently. Do not expect results to be similar. See some of the differences detailed in the table below.   Source: https://go.documentation.sas.com/doc/en/pgmsascdc/default/casstat/casstat_gammod_overview02.htm   The Generalized Additive Model node became available in SAS Model Studio in 2020.1.4. GAM is now included in the Advanced Template for an Interval Target in SAS Model Studio.     Defaults for the GAM node are: Selection Method: Boosting Interval target probability distribution: Normal Interval target link function: Identity Binary target link function: Logit Degrees of freedom: 4 Class input order: Formatted Class input coding: GLM as shown in the screen capture below.     Other distributions available are Gamma, inverse Gaussian, negative binomial, Poisson and Tweedie.     The default boosting options are: Maximum number of iterations: 500 Learning rate: 0.1   By default, smoothing plots are displayed as individual plots (up to 10 plots) and grouped as one report.     If you automatically generate a pipeline in SAS Model Studio, if will consider a GAM model.     You can, in fact, force it to include a GAM model in the pipeline, if you so desire.     Here is the resulting pipeline for my data (HEART data) when I forced it to include GAM.     With my data the GAM model did better than Forest or Linear Regression. However, Gradient Boosting and the Ensemble model did better than GAM.     Summary   Generalized additive models are useful when you have neither linearity nor normality.  SAS Model Studio makes it very easy to add a GAM model using the GAM node. SAS Model Studio now includes GAM in its Advanced Template with an Interval Target.  SAS Model Studio also considers GAM in its automatically generated pipeline if you have an interval target.   Because it is so easy to add a GAM model in SAS Model Studio, you can always include a GAM to see if it outperforms your other models.   For More Information SAS Viya Platform Programming Documentation Accuracy versus interpretability? With generalized additive models (GAMs), you can have both   Find more articles from SAS Global Enablement and Learning here. ... View more

In November 2022, ChatGPT took the nation by storm. It had over a million users within the first five days.   ChatGPT captured the world’s attention and propelled AI into mainstream conversations by everyday people, allowing them to witness the disruptive potential of AI first hand.  But what is it ChatGPT? How does it work? This post will explain it in layman’s terms.   ChatGPT is a generative AI chatbot.  GPT is the large language model (LLM) behind the scenes.  ChatGPT allows users to interact with the underlying model in a way that kind of scarily mimics interactions with a real human.   Is ChatGPT more than just a glorified search engine or Alexa on steroids? Actually, yes it.   ChatGPT and other generative AI tools can do a wide variety of tasks. For example, they can:   Write your essay on The Odyssey for your English class (but no guarantees that you’ll get an A) Run a calculation (no guarantee that results are correct) Generate some computer code (no guarantee you won’t have bugs) Have a conversation with you (no guarantee that everything ChatGPT tells you is true any more than when you shoot the breeze with your Uncle Charlie at a family gathering)   You may be beginning to see a theme here…no guarantees. But let’s set aside the topic of accuracy or quality for now. Additional examples of things that ChatGPT and other generative AI tools (backed by various LLMs) can do:   Predict protein structures Fill the shoes (figuratively) of some humans at customer support phone banks Generate images and videos from text or, conversely, generate text from images and videos Develop and tell stories to your children Write tweets and emails And of course, true to their roots, translate text from one language to another Even be a supportive friend?  See this advertisement for Pi, created by InflectionAI. Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Based on the question you provide, ChatGPT uses machine learning algorithms to attempt to understand the context of the conversation and generate what sound like appropriate responses. To produce a response, it predicts the next word in a given sequence based on patterns its learned from human language.  However, sometimes ChatGPT will simply string names and words together and confidently provide an answer that sounds coherent but in fact has no basis in reality. These are termed AI "hallucinations".   ChatGPT is available as an API that can be integrated into different applications and is embedded into Microsoft products.  Remember that ChatGPT backed by the LLM GPT is just one of many generative AI chatbots backed by other LLMs. LLMs can have billions of parameters and be trained on terabytes of text.  Many of them accomplish similar tasks, but some with niche focuses.   How Do Large Language Models Like GPT Work?   GPT stands for Generative Pre-Trained Transformer.  Let’s break that up:   Generative   LLMs are able to generate new text, images and videos that seem coherent and real. Generative AI is not just cutting and pasting.  It produces original content on demand. Rather than simply regurgitating searched information, generative AI creates new text, images, videos, etc. It can write original essays or create original art. It can even create virtual “people.”  For example, the generative AI WuDao 2.0 created an imaginary student Zhibing Hua who is tasked with learning all the material at Tsinghua University.  See this link to watch a supposed video of Zhibing Hua.   Pre-Trained   Unlike a search engine that searches any data, pre-trained LLMs like GPT are trained on a specific set of data over a specific time. For example, the ChatGPT released in November 2022 did not have information on any events or web content that occurred after 2021.  To train LLMs, web crawlers are used to pull hundreds of GB of data from books, articles, web pages, and social media on the Internet.  GPT-3 was trained on close to 500 billion words from millions of websites.  LLMs also adds new user inputs to their “knowledge”.   One of the tricks is differentiating fact from fiction. Open AI made a considerable effort to train GPT on credible sources. To accomplish this, they tried to only scrape web pages that had been curated or filtered by humans. ChatGPT was trained with both supervised learning and with reinforcement learning with humans ranking generated output.   Transformer   GPT is an autoregressive Transformer model. Transformer models are based solely on attention mechanisms.  The Transformer model inputs tokens.  It contextualizes each token with other (unmasked) input tokens using an attention mechanism.   To understand this better, let’s explore a little history of language models that led us to where we are today.   History   The roots of the deep learning techniques that have enabled the advent of the modern LLMs lie in efforts to improve automated machine translations from one language to another. What is translation---in fact, what is language itself--if not a longing to connect?  An innate yearning to understand another being and in turn be understood by that being.   Natural language processing and word sense disambiguation are essential components of translation. Other tasks by LLMs include content generation, automated data annotation, completion prediction, and the more advanced tasks of reading comprehension, commonsense reasoning, and natural language inferences.   Many older language models were based on recurrent neural networks (LSTM or GRU) in an encoder-decoder configuration. (See my earlier post Recurrent Neural Networks in SAS Viya for more information about RNNs).  Over time, language models grew and grew.  See graphic showing language models over time by the number of parameters.    Source: https://cmte.ieee.org/futuredirections/2023/03/18/add-ernie-bot-to-the-list   In addition to increasing is size, the 2010s saw some tremendous breakthroughs in model architecture, most significantly, the move to Transformer architectures. Prior to the advent of Transformer architectures, natural language processing commonly used RNNs, such as LSTMs and GRUs.  Eventually, some of these older models also connected the encoder and decoder through an attention mechanism. An attention mechanism selectively focuses on parts of the source sentence during translation.   To understand the importance of attention mechanisms, let us first turn our attention to language translation. Some languages are fairly easily translated word for word, because the word order is similar. In other languages, the word order is quite different. Let’s pick on English versus German for example. A literal word-for-word translation does not work. One author illustrates this as follows.       Another good example of attention is illustrated in the following sentence.  Following the word "eating" a noun that is a food item is expected.       You can interpret attention as a vector of importance weights.  To predict a word in a sentence, you can use the attention vector to estimate how strongly it is correlated with (i.e., how well it "attends to") other elements.  Sum the values weighted by the attention vector as the approximation of the target.   It was discovered that not only was the attention mechanism helpful, it was actually all you need!  Suddenly, recurrence and convolutions were dispensed with entirely, and the Transformer architecture took over.  The Transformer architecture is able to take excellent advantage of parallel processing, thereby reducing compute time.  The Transformer model inputs tokenized (byte pair encoding) input tokens. At each layer, the Transformer contextualizes each token with other (unmasked) input tokens in parallel via the attention mechanism.   The Transformer architecture is the key to modern LLMs like GPT. Not only can the Transformer architecture be used for natural language processing, but it also works for computer vision and audio processing.   Note that these advances would not be possible without the ready availability of large scale datasets and cloud computing.   ChatGPT's Recent History Specifically   So let’s examine the recent timeline specifically of Open AI’s ChatGPT. ChatGPT was developed by OpenAI, an AI research laboratory that includes: Open AI Incorporated (non-profit) OpenAI LP (for profit subsidiary) Here’s a summary timeline: 2015 (December): Open AI launched as a non-profit. 2018: Elon Musk resigned his Open AI board seat, citing "a potential future conflict." 2019 (March 11): OpenAI formed the for-profit subsidiary OpenAI LP. 2019: OpenAI LP received a $1 billion investment from Microsoft. 2020 (May): The Large Language Model GPT-3 was released. 2022 (October 27): Open AI board member and largest funder Elon Musk finalized acquisition of the major social media company Twitter. 2022 (November 30): OpenAI released an early demo of ChatGPT and made it available to the public for free; the chatbot quickly went viral on social media. 2023 (January): ChatGPT reached 100 million monthly active users just two months after launch, making it the fastest-growing consumer application in history.  For comparison, TikTok took about nine months after its global launch to reach 100 million users, and Instagram took well over two years. This large user base provided valuable feedback to help train ChatGPT's responses. 2023 (January 23): Microsoft confirmed new multiyear investment of billions of dollars into ChatGPT. FYI, Microsoft Azure is the exclusive cloud provider for ChatGPT and, in fact, for all Open AI tools. 2023 (March 16): Microsoft announced that ChatGPT’s LLM is embedded in Microsoft 365 Copilot to help relieve us from the “drudgery of work.” 2023 (March 22): News media and social media reported on an open letter with famous signatories including Elon Musk asking to “Pause Giant AI Experiments.”  This generated additional buzz about ChatGPT and AI. 2023 (April 10): Open AI Chief Executive met with Japan’s prime minister Fumio Kishida. 2023 (July 26)  Evan Gold of StockMKTNewz reported on Twitter that Microsoft will supply Japan’s government with the technology underpinning its ChatGPT generative AI for use in clerical work and analysis.       ChatGPT WannaBes   Once ChatGPT went viral, other large tech companies such as Alibaba, Baidu, Google and Meta accelerated development of their own LLMs. Some of these, such as LlaMA by Meta, are autoregressive decoder models quite similar to GPT.  There are tons of chatbots and LLMs; below are listed just a few.         Many of these other models compare themselves to GPT, and claim they are better.  Below see Jurassic-1's comparison to GPT-3.       Baidu also claims that Ernie 3.5 surpassed both GPT-3 and GPT-4 in several benchmark tests.   Regulation   As with all areas of AI, regulations and norms are lagging behind the rapid pace of the technology's development. A couple examples of steps toward generative AI regulation by major world powers include:   The Cyberspace Administration of China has mandated security reviews for all generative AI-related services who wish to operate in China.  They also have declared AI providers responsible for the validity of the data they use to train generative AI tools. On July 22, 2023, representatives from a number of tech companies (e.g., OpenAI, Microsoft, Amazon, Anthropic, Google, Inflection AI, and Meta) met at the US White House in advance of a Presidential Executive Order to be released by President Biden.  The role of the tech companies was ostensibly to make commitments on measures to ensure AI safety.  These voluntary commitments allegedly include: Watermarking AI generated videos Thoroughly testing systems before release Investing in cybersecurity Dedicating resources to solve societal challenges such as climate change   Is SAS’s ChatBot application similar to ChatGPT?   No.  SAS's Chatbot is not currently integrated with GPT or any other modern LLM.  SAS is currently not investing in developing its own LLM, and it is not on the roadmap.  It is possible that some SAS products will be integration with LLMs in the future.   Conclusion   Has ChatGPT added fuel to the AI race?  Will improved generative AI in one organization ignite the spending of resources in other organizations to produce even better generative AI? Will big tech companies, startups, universities and governments all race to see who can develop better, faster, more comprehensive generative AI systems?   What is certain is that massive amounts of financial and energy resources are being invested in generative AI, LLMs and foundation models. Because generative AI is extremely energy-intensive, costs and carbon emissions are also being discussed and considered.   AI marketers are also very cautious to spin their messages to assure humans that AI is not here to take your job. They use terms like “AI co-pilots” and maintain that only the drudgery of work will be taken by AI, leaving the fun and creative stuff for humans.       Do you think that’s true?   I will leave you with a quote from Professor Wang Xiaogang, the Co-founder and Chief Scientist of SenseTime.  “AGI [Artificial General Intelligence] has given rise to a new research paradigm, which is based on powerful foundation models, unlocking new capabilities through reinforcement learning and human feedback, therefore efficiently solving open-ended tasks. AGI will evolve from a 'data flywheel' to a 'wisdom flywheel', ultimately leading to human-machine symbiosis.”   For More Information Neural Machine Translation by Jointly Learning to Align and Translate. 2014. Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio. Language Models are Unsupervised Multitask Learners. 2019. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Attention Is All You Need. 2017. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin. Effective Approaches to Attention-based Neural Machine Translation. 2015. Minh-Thang Luong, Hieu Pham, Christopher D. Manning. Language Models are Few-Shot Learners. 2020.  Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, Dario Amodei. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. 2019. Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, Bryan Catanzaro.  Large Language Model Evaluation in 2023: 5 Methods. 2023. Burak Ceylan. Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, the World’s Largest and Most Powerful Generative Language Model. Oct 11, 2021. Paresh Kharya and Ali Alvi.   Find more articles from SAS Global Enablement and Learning here. ... View more

The RNNSPEC object now lets you easily compare RNN models to other time series models in SAS Visual Forecasting’s ATSM Package!     RNNS can give highly accurate results when working with sequential data such as time series data.  They are used in many applications when order matters.  Examples include natural language understanding, natural language generation, physical processes, and, of course, forecasting. The network is called recurrent because the network feeds back into itself and makes decisions in several steps.  Some RNN models can remember dependencies for long periods of time.  For more background on RNNS, see my earlier post RNNs in SAS Viya which focuses on using RNNs with SAS Visual Data Mining and Machine Learning.     The RNNSPEC object generates time series recurrent neural network (RNN) model specifications.  The RNNSPEC object first became available in SAS Viya in the 2021.2.2 stable release for use in the TSM object.  It became easier to use with the ATSM package in LTS 2023.03 (May 2023).     ATSM Overview     Before we get into the RNNSPEC object, let’s do a brief review of the ATSM Package.  Recall that we use the ATSM package to AUTOMATICALLY create time series models.  The steps involved include:     Define time series variables to be used for modeling and forecasting Diagnose and generate time series models Set diagnose specifications for each model family in consideration Forecast time series Set model selection specifications Forecast time series with the champion model Collect results Save the outputs for review or other analysis tasks     We have four main types of objects in the ATSM package:     Time series modeling and forecasting objects Objects for controlling model identification Collector objects Repeater objects Lets look more deeply at each of these object categories:     Time series modeling and forecasting objects.  These include the main forecasting engine object, the FORENG object.     Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Objects for controlling model identification.  These include the objects that control model identification.       Collector objects collect results.  They essentially create a snapshot of results from the ATSM objects and save the results to a CAS table.         Repeater objects relay information.  They make information from a CAS table (previously defined by a collector object) available to other ATSM objects.         So how do these all fit together?  In this partial diagram, we see that the collector object OUTDIAG collects the parameters (diagnostic control options) from the DIAGSPEC object instance. The repeater object INDIAG replays (repeats) the diagnostic control options to the DIAGNOSE object instance.         Here see a full ATSM data flow example.       Source:  https://go.documentation.sas.com/doc/en/pgmsascdc/v_037/castsp/castsp_atsm_sect002.htm     The FORENG (forecasting engine) object is the brains of the process.  This is where the modeling happens.  The addFrom method adds models to a FORENG instance.  The model specification objects available in the addFrom method are shown in the table below.         How do we use these objects?  To use ATSM objects within PROC TSMODEL your code must: Require the ATSM package in the TSMODEL procedure Declare each object referenced in your programs Initialize the object to create an object instance Use object methods to set options or configure inputs and outputs Organize and customize workflows using scripting language statements     Using the RNNSPEC object in SAS Visual Forecasting     Now that we’ve had a brief background primer, let’s talk specifically about the RNNSPEC object.     Three RNN model architectures are available in SAS Visual Forecasting Original recurrent neural network (RNN) Long short-term memory unit network (LSTM) Gate recurrent unit network (GRU)     These are available via PROC TSMODEL Time Series Neural Network Forecasting Package standalone RNN forecasting package available beginning in stable release 2020.1.2 TSM package includes RNNSPEC ATSM package now supports RNN forecasting models available beginning in LTS 2023.03 (May 2023)   The RNNSPEC object can now be used in the ATSM package to let you compare RNN models to your other forecasting models such as autoregressive integrated moving average models, exponential smoothing models, and unobserved components models.  You now have the same options in TSM and ATSM.  You must first define the RNNSPEC object using the TSM package and the ATSM package then honors all RNNSPEC object options.     You can add RNN models directly to a ATSM FORENG object, an ATSM SELSPEC object or a TSM FORENG object.  The RNNSPEC must be initialized in the TSM Object.  See a simple flow diagram below.       Source:  SAS Viya documentation https://go.documentation.sas.com/doc/en/pgmsascdc/default/casecon/casecon_tsm_sect127.htm       As with your other forecasting models, you can include input and event variables in the model by using the TSDF object.  Use inputs and events from the Time Series Data Frame (TSDF) object in an RNN model by using the RNNSPEC object’s AddTF method. The RNNSPEC model output can be viewed in the collector objects OUTFOR, OUTMODELINFO, OUTSELECT, and OUTSTAT.         Below is an example of code for running RNNSPEC with DIAGNOSE object SPECS courtesy of Taiyeong Lee.      /*******************************************/ /* */ /* RNNSPEC with DIAGNOSE object specs */ /* based on example from Taiyeong Lee */ /* modified 20230622 by Beth Ebersole */ /* */ /*******************************************/ /* Create a CAS session and a CAS library */ cas mySession sessopts=(caslib=casuser timeout=1800 locale="en_US"); libname mycas cas sessref = mySession; data mycas.air; set sashelp.air; run; proc tsmodel data=mycas.air outlog=mycas.outlog logcontrol=(error=keep warning=keep note=keep none=keep) outobj=(outstat=mycas.outstat modInfo=mycas.modInfo airSelect=mycas.airSelect); id date interval=month; var air; require atsm tsm; SUBMIT; declare object dataFrame(tsdf); rc = dataFrame.Initialize(); rc = dataFrame.AddY(air); rc = dataFrame.SetOption('seasonality', 12); declare object diagSpec(diagspec); rc = diagSpec.Open(); rc = diagSpec.SetARIMAX(); rc = diagSpec.SetUCM(); rc = diagSpec.Close(); declare object diagnose(diagnose); rc = diagnose.Initialize(dataFrame); rc = diagnose.SetSpec(diagSpec); rc = diagnose.SetOption('holdout', 12); rc = diagnose.Run(); declare object rnnspec_GRU(rnnspec); rc = rnnspec_GRU.Open( ); rc = rnnspec_GRU.SetOption( 'RNNTYPE','GRU', 'NINPUT', 12, 'NVALIDATION',12, 'NLAYER', 4, 'NNEURONH',30, 'NORMALIZE', 'STD', 'BATCHSELECTION', 'SEQ', 'POSTTRAIN', 'YES', 'SEED', 12345 ); rc = rnnspec_GRU.SetOptimizer( 'ALGORITHM','ADAM', 'LEARNINGRATE', 0.1, 'BETA1', 0.8, 'BETA2', 0.9, 'LEARNINGPOLICY', 'STEP', 'STEPSIZE', 5, 'GAMMA', 0.5, 'STAGNATION', 20, 'MINIBATCHSIZE', 8, 'WARMUPEPOCHS',15, 'MAXEPOCHS', 100 ); rc = rnnspec_GRU.Close(); declare object rnnspec_LSTM(rnnspec); rc = rnnspec_LSTM.Open( ); rc = rnnspec_LSTM.SetOption( 'RNNTYPE','LSTM', 'NINPUT', 12, 'NVALIDATION', 0, 'NLAYER', 4, 'NNEURONH',30, 'NORMALIZE', 'STD', 'BATCHSELECTION','SEQ', 'POSTTRAIN', 'NO', 'SEED', 12345 ); rc = rnnspec_LSTM.SetOptimizer( 'ALGORITHM','ADAM', 'LEARNINGRATE', 0.01, 'BETA1', 0.8, 'BETA2', 0.9, 'LEARNINGPOLICY', 'STEP', 'STEPSIZE', 5, 'GAMMA', 0.5, 'STAGNATION', 20, 'MINIBATCHSIZE',8, 'WARMUPEPOCHS',15, 'MAXEPOCHS', 100 ); rc = rnnspec_LSTM.close(); declare object foreng(foreng); rc = foreng.Initialize(diagnose); rc = foreng.addFrom(rnnspec_GRU); rc = foreng.addFrom(rnnspec_LSTM); rc = foreng.SetOption('lead', 12); rc = foreng.SetOption('alpha',0.05); rc = foreng.setOption('SEASONTEST','NONE'); rc = foreng.Run(); declare object modInfo(outmodelinfo); rc = modInfo.Collect(foreng); declare object airSelect(outselect); rc = airSelect.Collect(foreng); declare object outstat(outstat); rc = outstat.Collect(foreng); ENDSUBMIT; print outlog; quit; proc print data=mycas.modInfo; run; proc print data=mycas.airSelect; run; proc print data=mycas.outstat; run;     Example Output:         Example Results:         For More Information SAS Viya documentation Recurrent Neural Networks:  An Essential Tool for Machine Learning  2018 by Yue Xi (with assistance from Doug Cairns) 2021 Post How to incorporate Recurrent Neural Networks in your SAS Visual Forecasting pipelines General background on RNNs by Brandon Rohrer To learn about Distributed and Parallel Hyperparameter Tuning of RNN Forecasting Models in PROC TSMODEL see this post 2023 by Taiyeong Lee. ... View more

RNNS are useful in working with any data where order matters. Examples include: Natural Language Understanding (NLU) and Natural Language Generation (NLG), for example “Beth drank coffee” is not the same as “Coffee drank Beth.” Forecasting, for example, forecasting profits Physical processes, for example, erosion Image captioning, where the input is an image, and the output is text This blog will provide a little background, describe the ways you can create RNNs in SAS Viya, and provide example code.   RNNs are called recurrent because the network feeds back on itself. RNNs are built with Long Short-Term Memory (LSTM) models or gated recurrent units (GRUs)  One advantage of LSTM models is that they can “remember” dependencies for long periods of time.   This process is explained in detail in an excellent 30-minute video by Brandon Rohrer. If you have time, watch the video. If not, the next 3 images provide a quick synopsis.     Source: youtube.com/watch?v=WCUNPb-5EYI   A recurrent neural network starts with new information which runs through a neural network. Then a “squashing” function (activation function, link function) is used to map the output of a neuron to a limited range. I don’t know who first coined the term squashing function, but I love it. Very descriptive. You are essentially taking any real number and squasing it down to a specific limited range. Commonly used squashing functions are:   Sigmoid function, which maps the output to the range of 0 to 1 Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.       Hyperbolic tangent, which maps the output to a range of -1 to 1    Both of these functions are used in the full RNN LSTM process. Hyperbolic tangent is commonly used to squash the output from a neural network to a number from -1 to 1. This number from -1 to 1 is now your prediction and can also be fed back into the neural network again as shown below.     Source: youtube.com/watch?v=WCUNPb-5EY   A recurrent neural network uses long short-term memory process to make decision in multiple steps. The essential function for making these decisions (selection, forgetting, and ignoring) is a gate. Mathematically, the gate is simple a multiplication function where the output of the previous step is multiplied by a number. Multiplication times zero is a closed gate; that output is stopped right there. Multiplication by 1 is a fully open gate; the output advances fully intact. You can also have a halfway open gate by multiplying by 0.5, and so on.   An RNN with LSTM has: PREDICTION BRANCH: From new input information, an neural makes a prediction squashed between [-1,1] by a hyperbolic tangent function, then passed on to the next phase IGNORING BRANCH: Sets aside irrelevant information, so it doesn’t affect the other predictions. FORGETTING BRANCH: Adds the predictions to the information coming from the first gate. The neural network and the squashing function decide what information will be forgotten and what information will be kept. Then the multiplication junction (gating) determines how much of the information is going to pass to the next phase. SELECTION BRANCH: Both new information and previous information are inputs. This neural network selects which memory and collected possibilities will be considered to make new predictions. This is illustrated conceptually below:     Source: youtube.com/watch?v=WCUNPb-5EYI   RNNs in SAS Viya   SAS Viya has two CAS action sets that allow you to build, train, score, and export RNNs.     The deepRNN action set is designed specifically for RNNs. It uses one of two loss functions: For classification, an entropy loss function For regression, normal loss function The deepRNN action set includes 3 actions as shown below.     The deepLearn action set is highly flexible and lets you build many types of neural networks. With respect to RNNS, the deepLearn action set supports both long short-term memory (LSTM) and gated recurrent unit (GRU) RNNs.       RNN Coding in Python Using SWAT to Run CAS Actions   To build your own RNN layer by layer using the deepLearn action set, you would first start a session and load the action set.     sess = swat.CAS('sas-cas-server-default-bin', portnumber, 'gatedemoXXX', 'lnxsas', caslib="casuser") sess.loadactionset('deepLearn')     Then you would build an LSTM model (here the model is named classifer)     sess.buildmodel(model=dict(name='classifier', replace=True ), type='RNN' )     Next add an input layer     sess.addlayer(model='classifier', name='data', layer=dict(type='input') )     Add your RNN layers     sess.addlayer(model='classifier', name='rnn11', srclayers=['data'], layer=dict(type='recurrent', n=50, init='xavier', rnnType='LSTM', act='TANH', outputType='samelength', reverse=False ) ) sess.addlayer(model='classifier', name='rnn21', srclayers=['rnn11'], layer=dict(type='recurrent', n=50, init='xavier', rnnType='LSTM', act='TANH', outputType='samelength', reverse=False ) ) sess.addlayer(model='classifier', name='rnn31', srclayers=['rnn21'], layer=dict(type='recurrent', n=50, init='xavier', rnnType='LSTM', act='TANH', outputType='encoding', reverse=False ) )     Add your output layer; use a softmax activation function.     sess.addlayer(model='classifier', name='outlayer', srclayers=['fc1'], layer=dict(type='output', act='SOFTMAX' ) )       (The above code is from SAS Viya documentation )   Recall that the softmax function takes a vector of scores and transforms it to a vector of values between 0 and 1 that sum to 1.           For example:     RNNs on GPUs   SAS supports using GPUs for recurrent neural networks to reduce processing time, but keep in mind that the underlying algorithm is slightly different when run on GPUs than when on CPUs   There are a number of requirements, including: The RNN can have only a single input layer and a single output layer Recurrent layers must be either unidirectional forward layers or bidirectional layers All recurrent layers must use the same memory cell type (LSTM, GRU, RNN, or PASSTHROUGH) All recurrent layers must have the same hidden dimension Text generation with RNNs on GPUs is not supported Dilated RNNs Available via SAS Viya dlModelZoo   SAS Viya dlModelZoo provides a variety of predefined PyTorch models that can be used OOTB, including dilated RNN. Dilated Rnns are designed to help address RNN training issues, such as complex dependencies, vanishing gradients, and exploding gradients. Dilated RNNs are also easily parallelized, and may require fewer parameters.   For More Information SAS Viya documentation Blog on Recurrent neural networks: An essential tool for machine learning, June 7, 2018 by Yue Xi (with assistance from Doug Cairns) Blog on Distributed and Parallel Hyperparameter Tuning of RNN Forecasting Models in the TSMODEL Procedure, April 12, 2023 by Taiyeong Lee. Blog on Using the dlModelZoo action set to import PyTorch models into SAS, April 11, 2023 by Scott Pope. How LSTM and RNNs work, ~ 2018 by Brandon Rohrer.   ... View more

Background   Issues with biased data collection and bias magnification by algorithms has plagued AI since its inception. Biased collection has been well understood since the study of statistics began. For example, data collected only on one segment of the people in one country may not extrapolate well to the full population of 7.9 billion people on earth.   Historic biases can also be perpetuated through automated algorithms. Here’s one example.   The Diner’s Club cardboard credit card started in the United States in 1949. It was only available to a select group of 200 white men, essentially friends of the founders. Two years later the membership had grown to 42,000. Plastic credit cards issued by banks came out in 1958. Yet it wasn’t until mid 1970s that women and minorities were able to get credit cards on their own. But of course, at that point, it was still much more difficult for them to get credit because as a group they lacked any credit history. Formulas that considered credit history were biased against those who had not even been permitted to have a credit history.   SAS Viya has been able to assess bias for a while. And even better, SAS Viya can now help you mitigate bias inside your modeling process! SAS Viya lets you:   Assess a sensitive variable for bias using the SAS Model Studio interface Assess a sensitive variable for bias using by programming in CASL, Lua, Python or R Mitigate bias using by programming in CASL, Lua, Python or R (NOTE: It is on the Roadmap for this capability to be added to SAS Model Studio also!) Explore your data to find bias using SAS Visual Analytics   Understanding the Fairness Metrics Used to Assess and Mitigate Bias in SAS Viya   The plots on the Fairness and Bias tab in the Results section highlight potential differences in model performance for different groups within specified sensitive variables.   The assessBias action provides a number of useful measures of bias. These include performance bias, performance bias parity, prediction bias, and prediction bias parity.   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   The mitigateBias action addresses bias using the exponentiated gradient reduction (EGR) algorithm. EGR runs multiple iterations. In each iteration it can reweight and relabel the data, and then trains a new classification model. So it actually incorporates fairness constraints within the modeling process!   The SAS Viya mitigateBias action with EGR supports a number of fairness measures: demographic parity, equalized opportunity and equalized odds. It will take these measures into account in finding your best model, and will provide you detailed results of not only the model performance but also the bias metrics.     Let’s explain some of the fairness statistics used to assess and mitigate bias by using a simple example. Say we are using an algorithm to determine hiring rates. We want to check for any bias related to whether someone is Latinx or not. Let’s consider a binary sensitive variable LatinxStatus, which can be either LATINX or NONLATINX.   Performance bias parity compares the fit statistics of the model for the LATINX group versus for the NONLATINX group. If the model is not a good fit for both groups, that is a problem. Try some other models to find one that works well for both groups. It’s easy to compare models with SAS Viya, the fastest, most productive AI and Analytics.   Prediction bias parity compares the probability of hire between the LATINX and NONLATINX groups. The measure is the difference between those two probabilities. This should be low; the best model would not show a different average predications among goups if the sensitive variable (or its surrogates…but that’s a topic for another day) does not play a role in predicting hiring.   Demographic parity compares the selection rate of each category (LATINX and NONLATINX) of the sensitive variable LatinxStatus. The measure is the difference between those selection rates. Demographic parity can help an organization balance out historical biases that impact the data.   Equalized opportunity compares the true positive rate for the LATINX group versus the NONLATINX group. Be sure you are getting a high true positive rate for each group.   Equalized odds looks at the maximum difference in true positive rate OR false positive rate between the NONLATINX and LATINX groups. You want to be sure there that both the TPR and FPR are similar between the two groups.   Watch how this works in action in SAS Viya:   To see a demonstration of assessing bias using programming, watch my December 2021 YouTube SAS Fair Artificial Intelligence Tools. To see an example of assessing bias using SAS Model Studio (pipeline interface) and mitigating bias using CASL in SAS Studio, watch my April 2023 YouTube mitigateBias with SAS.   The screen capture below shows how sensitive variables can be designated in the Data pane of SAS Model Studio.     The screen capture below shows example results of the assessBias action with graphs of performance bias and performance bias parity, prediction bias and prediction bias parity, and bias metrics and bias parity metrics.     CODE examples:   Here is the code I used with screen shots of the results interspersed:   /* BethHeartSexMITIGATE */ cas MySession sessopts=(caslib=casuser timeout=1800 locale="en_US"); libname casuser cas caslib="casuser"; proc casutil; droptable casdata="casuser.heartone" quiet; load data=sashelp.heart outcaslib="casuser" casout="HEARTone" promote; run; quit; proc contents data=casuser.heartone; run; proc print data = casuser.heartone (obs = 100); run; data casuser.hearttwo; set casuser.heartone; where DeathCause = "Cancer" or DeathCause = "Coronary Heart Disease"; run; data casuser.heartthree; set casuser.hearttwo; if DeathCause = "Coronary Heart Disease" then DeathCause = "Heart"; run; proc print data=casuser.heartthree (obs = 50); run;       Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Train a Gradient Boosting Model   proc cas; decisionTree.gbtreeTrain / inputs={"Systolic", "Diastolic", "Weight", "Height", "Sex"}, maxLevel="5", saveState={name="gbtreeASTORE", replace="True"}, seed=1234, table="HEARTthree", target="DeathCause"; run;       Assess Bias   proc cas; fairAITools.assessBias / modelTable="gbtreeASTORE", modelTableType="ASTORE", event = "Heart", predictedVariables={"P_DeathCauseCancer", "P_DeathCauseHeart"}, response="DeathCause", responseLevels={"Cancer", "Heart"}, sensitiveVariable="Sex", table="HEARTthree"; run;   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.         Mitigate Bias   proc cas; fairAITools.mitigateBias / biasMetric="DEMOGRAPHICPARITY", event="Heart", learningRate="0.01", maxIters="10", predictedVariables={"P_DeathCauseCancer", "P_DeathCauseHeart"}, response="DeathCause", responseLevels={"Cancer", "Heart"}, sensitiveVariable="Sex", table="HEARTthree", tolerance="0.005", trainProgram=" decisionTree.gbtreeTrain result=train_res / table=table, weight=weight, target=""DeathCause"", inputs= { ""Systolic"", ""Diastolic"", ""Weight"", ""Height"", ""Sex"" }, nominals={""DeathCause"",""Sex""}, nBins=50, quantileBin=True, maxLevel=5, maxBranch=2, leafSize=5, missing=""USEINSEARCH"", minUseInSearch=1, binOrder=True, varImp=True, mergeBin=True, encodeName=True, nTree=15, seed=12345, ridge=1, savestate={ name=""HEART_gb_astore"", replace=True } ; astore.score result=score_res / table=table, casout=casout, copyVars=copyVars, rstore=""HEART_gb_astore"" ; ", tuneBound="True"; run;     Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.       A number of additional code samples are available in the documentation.     These code examples allow for various combinations of feature (input) and target (outcome) types, as shown in the slide image below, and repeated as text for anyone who cannot read the image:      Assess bias: For nominal target For interval target Using DATA step or DS2 scoring Using prescored data and reference level With no response variable Mitigate bias: Mitigate demographic parity for a binary target with a binary or nominal sensitive input Mitigate equal opportunity for a binary target with a binary or nominal sensitive input   Always Always Always Explore Your Data   In this blog I’ve shown you how to assess and mitigate bias as part of the modeling process using coding and SAS Model Studio. But remember how much you can learn about your data and how much time and headaches you can save yourself down the road if you first explore your data using SAS Visual Analytics. Here’s an example using the same HEART data I used in the mitigateBias video I pointed you to.       Release History   The assessBias action became available via programming (in CASL, Lua, Python or R) in October 2021. The following month (November 2021) the Fairness and Bias tab became available in SAS Model Studio. Most recently, in October 2022, the mitigateBias action became available via programming (in CASL, Lua, Python or R). This information is also shown in the table below.       FOR MORE INFORMATION Videos   Tamara Fischer and Veronique Van Vlasselelaer’s webinar on Creating Fair Machine Learning Models Hiwot Tesfaye Responsible AI in Practice using SAS Visual Analytics (length 22 minutes) Joe Madden SAS Viya 2022.10 Release October 2022 (very brief overview)  start minute 1:57   Blogs Jagruti Kanjia and Ricky Tharrington’s blog Assess model bias in SAS Viya 4 May 13, 2022 Suneel Grover’s blog AI/ML Bias Detection and Mitigation in Customer Analytics   Find more articles from SAS Global Enablement and Learning here. ... View more

As SAS CEO Jim Goodnight posted on 28 January 2023 on LinkedIn, “SAS is committed to developing, implementing and promoting trustworthy AI systems that help ensure sustainable improvements and responsible innovation for our customers, the economy and society.”           Specifically, SAS has committed to six trustworthy AI principles, as shown in the graphic below from the SAS web site:     Human-centricity Inclusivity Transparency Accountability Robustness Privacy & Security Human-centricity Human-centricity indicates that human well being and human agency must remain first and foremost. Not exactly like the Hippocratic Oath by doctors “First, do no harm,” but similar. Any AI that does nothing for the benefit of humans (and by extension the health of the planet we live in) should be questioned as to what is its purpose. As part of human centricity is the tenet of equity. AI should improve equity and should not exacerbate inequities.             Inclusivity AI should be fully inclusive. This means that it must ensure accessibility for all. It also means that its developers and managers should include a diverse set of perspectives and experiences. SAS is committed to maintaining a diverse, inclusive, equitable and satisfied work force. We also work diligently to ensure that our software is accessible. See my article on Accessibility.          Accountability Anyone who creates AI or builds applications that use AI will not be able to 100% control the irresponsible use of that AI. Therefore, to be responsible we must proactively identify adverse impacts and mitigate those impacts. To identify adverse impacts, we must first decide what impacts are adverse versus beneficial. See my previous blog on AI Ethics Part 1: Bots to read a discussion of this. The best surrogate that we have for ethics is laws. Unfortunately, laws surrounding AI are sadly lagging behind the AI technology itself. Nonetheless, we must make ourselved aware of and in compliance with laws that exist and also be engaged in guiding those conversations. SAS leadership is involved at highest levels of US AI ethics conversations. For example, Reggie Townsend, Director of the SAS Data Ethics Practice, serves on the US National Artificial Intelligence Advisory Committee. This committee advises the US President Biden and the US National AI Initiative Office on topics related to the development and use of AI.       Transparency Transparency is important. We must openly communicate the intended use and potential risks of AI applications. In addition, how the AI makes decisions such as denying loans or identifying possible fraudsters must be as transparent as possible. In some cases explainability is legislated, as in the financial sector. SAS Viya provides interpretability aids out of the box, including:   LIME (Local Interpretable Model-agnostic Explanations) ICE (Individualy Conditional Expectation) plots Partial dependence plots Variable importance plots and rankings Shapley values See my earlier blog on interpretability tools in Viya         Robustness The principal of robustness means that AI applications should operate safely and reliably. It also enables mechanism to assess and manager risks throughout a system’s life cycle. SAS Viya includes a number of ways to manage algorithms and models and to monitor their depreciation over time.              Privacy & Security Privacy and security means that SAS software must protect any data it accesses as well as any IP of its users. SAS has close to 50 years of experience in software privacy and security and continues to follow the latest in this realm.               For More Information: See the excellent webinar on Trustworthy AI and using SAS tools to mitigate bias by Tamara Fischer and Veronique Van Vlasselaer. Ponder This: Grannies and Grammys vs AI Is all the AI in the world equal to the wisdom of one granny?  AI might have pointed to Beyonce to win album of the year (she didn't). AI might have pointed to Taylor Swift for getting called up on stage at least one time (she didn't). But it took a wise Granny to predict that Harry Styles would win album of the year.  Harry was not the greatest singer nor did he have the greatest songs.  But Harry has that certain je ne sais quoi that eludes quantification and thus AI.  But any Granny sees it immediately and tells her young granddaughter, “Stay away from that guy; but if you must invite him for dinner, seat him next to me.”   Harry Styles hugs a granny who predicted his Grammy win in February 2023   ... View more

AI is a tool. Just like a hammer is a tool. A hammer can be used for many things. It can be used to build a beautiful house or a fun tree fort for kids to play in. Or. A hammer can be used to destroy property or to break a bone.   AI can be used for many positive things, like identifying cancer cells and improving vehicle safety. Or it can be used for blackmail, revenge, and spreading misinformation among other disturbing things.   This blog will talk specifically about Bots. On the one hand, AI is used to create and use more and more bots. On the other hand, AI is being used to identify and stop bots. Hmmm… does this mean that more and more AI must be combatted with more and more AI?   AI and Ethics   Ray Greenwood of SAS contends that there is no responsible or irresponsible AI any more than there is a responsible or irresponsible calculator. You can use a completely accurate calculator to cheat on your taxes, for example. It’s not the calculator’s fault and, to be honest, the calculator couldn’t care less. See Ray’s excellent webinar (starts at minute 10).   However, AI can be used responsibly or irresponsibly. And any mistakes, such as bias, in an algorithm, can be multiplied and magnified with AI.   One of the most challenging aspects of AI ethics is the ethics part. What one person believes is good, another may believe is evil. Is making weapons of war more accurate good or evil? Is improving the marketing of unnecessary, expensive products to low-income purchasers good or evil? Is replacing humans doing menial jobs good or evil? Is facial recognition and location software creating a benefit to society by helping to find criminals or is it an invasion of privacy?   With the caveat that the opinions expressed here are solely my own, here are some of the positive things that AI can accomplish:   Diagnosing diseases Finding and fixing mistakes Rooting out fraudsters Improving agricultural and industrial operational efficiency Finding the shortest path, saving time and energy Improving motor vehicle safety, saving lives Quickly identifying and locating school shooters Assessing drug and vaccine efficacy Entertainment Gaining supply chain efficiency and security Reducing emergency responder response times Intelligent traffic signals reducing gridlock But AI can also be associated with negative actions:   Deception, such as spreading misinformation for political campaigns or political objectives Fostering device addiction Blackmailing or embarrassing people with private information Providing biased results Lending credibility to bogus and inaccurate results Heavy electric power usage Use of resources in limited supply (e.g., rare earth elements) Creating revenge porn Creating fraud Lack of transparency “Stealing” jobs from humans   Although I am categorizing these tasks as positive or negative, there are many gray areas! Things are not always black and white. Would you move some of these items from positive to negative or vice versa?   Because there are many aspects of AI, I will focus in this blog specifically on bots.   Robots in Science Fiction   In Science Fiction, robots have often been anthropomorphised. Not only are they often given human emotions and agency but also given physical features like faces, arms, and legs.   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Often, through they course of the story, bots are determined to be either benevolent or nefarious. Rarely do you encounter the robot that just simply makes mathematical mistakes resulting in the doom of humankind. Instead they intentionally turn on humans, their creators, and willfully wreak havoc. It makes for a more exciting movie than say, a computer that denies a loan through a small bias that is magnified through its algorithm.     But in reality bots do not need faces or fingers (nor do they feel emotions or guilt or shame). Let’s talk about real present-day bots and how they fit into our modern world where so many transactions are done online.   Benevolent Bots   Some bots are designed to improve a user's experience online. For example, they may be programmed to provide the user with near real-time updates on topics of interest, like how many times the Dallas Cowboys kicker has missed the extra point in this one playoff game alone. Others are designed to answer user questions without spending the money on a real staff person. However, even some of these “helpful” bots can be mighty annoying when they pop up without a request or they can’t seem to figure out what you’re asking for.     Bots: Fake Accounts   Fake accounts that function automatically are also referred to as bots. Some of these may be harmless, but others are willfully used for illegal or unethical tasks.   These bots can be used to:   Commit fraud Abuse bonus offers or discounts Influence reviews of people or products Wreak havoc and create dischord Distribute misinformation and propaganda Spread malware Create more bots Romance scams (aka “catfishing”)   Even though Facebook removes billions of fake accounts every year, they still estimate that about 5 percent of their accounts (around 90 million) are fake. Spam bots attempt to mimic live user activity to spread content. They are common on all social media platforms and most of these spam bots require little or no human involvement and operate extremely efficiently.   Bots: Fake Reviews    Companies like FakeSpot use algorithms to help spot fake reviews on major webites like Amazon and Walmart. Tips for spotting fake reviews include:   The product has almost all five-star reviews There are bunch of reviews on the same date Wording is similar to other reviews Poor grammar and spelling mistakes Lack of detail Customers also bought section contains unrelated products Explaining away cons Overly flowery Review is 100% positive or 100% negative; people typically do not 100% love or hate a product or service. More savvy will be a fantastic review with one small negative that isn’t a deal breaker The review is very negative but mentions a competing product in glowing terms; this is called customer jacking   The following looks like a fake review to me.  Who reviews halogen light bulbs by saying "They where easy to install."  Ummm.  Wouldn't that be all light bulbs?     If you have time, check out a suspicious reviewer’s profile and their other reviews. If you see they repeat the same phrases in other reviews, or all their reviews are five stars or one star, that is a red flag. But really… who has time for this!?   Service Bots: Stealing Jobs?   Many restaurants now let place your food orders from your phone. A McDonald’s near Ft. Worth, Texas unveiled a new automated dining concept, targeted at carry-out customers, that takes it one step further and uses contactless order pickup as well as automated ordering. Perhaps this can help with reducing the spread of disease through contactless pickup? Or is this stealing menial jobs and as such a bad thing?     Other examples of bot uses include changing survey results, laundering money, or abusing free services. An example of the latter would be using a free cloud computing account offer to mine cryptocurrency. How are fake accounts created?   One solution to address the “stealing” of human jobs by AI and automation is to shorten the work week. The five-day, 40-hour work week has been the default standard in the United States since President Franklin Roosevelt signed the Fair Labor Standards Act of 1938. In the mid 1900s a full work week for Maryland state employees was 35.5 hours; the work week had been lowered over the years in lieu of pay raises. Perhaps forward-thinking companies that value employee satisfaction will break from the current tradition and launch a new full-time 32-hour four-day work week as its standard?   Bots: Stealing Tickets   In the now infamous Taylor Swift TicketMaster fiasco, shortly after her concert tickets went on sale the whole system crashed. According to Joe Berchtold of TicketMaster’s parent company “an onslaught of bots that crowded out real fans and attacked Ticketmaster’s servers. While the bots failed to penetrate our systems or acquire any tickets, the attack required us to slow down and even pause our sales. This is what led to a terrible consumer experience." One Ticketmaster detractor responded “for the leading ticket company not to be able to handle bots is, for me, an unbelievable statement. You can’t blame bots for what happened.” The lesson from this is that any business that doesn’t want its name dragged into the mud and its CEO dragged before the US Congress, better be bot-ready! https://www.nytimes.com/2023/01/24/arts/music/ticketmaster-taylor-swift-bot-attack.html   Spotting and Protection From Bots   • Use AI to combat AI! Protection against bots uses AI-based real-time analytics to make near instant authorization decisions. These analytics commonly look for anomalous activity. • Make it difficult for bots to create or access accounts. We’ve all seen the 9 pictures with “Are you a real person? Which of these contain fire hydrants?”   The trouble is, the trickier the protection gets the trickier the bots get leading to something of a vicious spiral.     Spam Bots   Even if they are spreading truths and not spreading disinformation, spam bots can be annoying and suck up your time and energy. Bots have been extremely effectively used to manipulate public opinion, distort financial markets, interfere in elections, and inflate the popularity of entertainers and politicians. In 2016, Twitter identified over 50,000 Russian-linked spam accounts that were cleverly designed to microtarget Twitter users and sow seeds of division related to the US election. Spam bots have also been used to spread misinformation about COVID-19, other diseases, and vaccines.   Phishing Attacks   Hopefully you have all taken the SAS security trainings, and mentally armed yourselves against phishing attacks. As we better arm ourselves, however, the phishers become more and more sophisticated. Dastardly bots can cleverly masquerade as friends/acquaintances or legitimate businesses and suck unwary people into providing confidential information or passwords.   Bots in Warfare    "Future wars may depend as much on algorithms as on ammunition,” according to Robert Work, the former US deputy secretary of defense. The US Defense Department and other defense departments continue to invest more and more in artificial intelligence. Work (who currently serves on the board of Govini, a data and analytics company) maintained in 2017 that “rapid advances in artificial intelligence — and the vastly improved autonomous systems and operations they will enable — are pointing toward new and more novel warfighting applications involving human-machine collaboration and combat teaming. These new applications will be the primary drivers of an emerging military-technical revolution.” Source:  Washington Post.     Folks like Work maintain that AI is transforming war just as earlier inventions like the rifle, telegraph, railroad, and airplane have done in the past. Big players in this arena include the usual “beltway bandits”, i.e., large US federal technology contractors like CACI, Leidos, Lockheed Martin, Northrop Grumman, Raytheon, and SAIC. In today’s military, AI is even used in soldier training, where soldiers experience simulated battlefields using virtual reality.   There has been much debate and discussion about the use of lethal autonomous weapons systems. These are sometimes referred to in the media as “slaughterbots,” “killer bots,” or "predator drones" These are weapons systems that use artificial intelligence (AI) to identify, select, and kill human targets without human intervention. The former US President Obama received much criticism as well as some applause for his use of lethal drones. Military drones are currently being used by both Ukraine and Russia in the Ukraine-Russia war.   Flying drones can be used for audio-visual reconnaissance or they can also be outfitted with small bombs. Electromagnetic pulses and other communication jamming/intercepting means can be used to disorient drones, interrupt or intercept communications between drones and their operators, or even co-opt the drone’s operation.   So the question becomes, if society agrees that having human soldiers kill other humans is a necessary and acceptable approach to international conflict, what is it about using AI to accomplish this same task that makes it no longer acceptable? Is it the fear of error? Is it because the AI lacks a conscience? Now that a human being is no longer pulling the trigger or dropping the bomb from a plane, is it somehow less ethical? Is it the fear that if there is an error, there will be no one to blame? The current international legal framework is woefully inadequate to address this topic or, frankly, most topics surrounding AI ethics.   While killing machines get most of the media attention, there are also ongoing cyberwars on many fronts with bots trying (and succeeding) to invade critical infrastructure and communications systems.       Note in the screen shot above that even while I searched online for information for this blog, I was microtargeted by AI with Cole Haan ads.   Who is to Blame?   One of the key ethical questions is who is to blame when something goes awry. Anyone who has ever driven a vehicle knows that in some situations an accident cannot be avoided. Even the best autonomous vehicles can unfortunately have some accidents, and lives or limbs will be lost. In that case, who is to blame? The owner? The manufacturer? The software? Yet another question that is unsettled in the established legal and ethical realm.   Conclusions   Once concern about AI is the fantasy of infallibility. Nothing is 100% accurate. Even if results are 99% accurate, 1 out of 100 times they are wrong. Some falsely believe that if an answer is given by AI, it must be unbiased and accurate. This is not true! Remember, garbage in, garbage out. Biased data leads to biased results. Inadequate data leads to inadequate results. Small errors can be magnified by AI. See My YouTube video on bias and SAS’s Fair AI Tools for more information on that topic.   Do you agree that there is no responsible or irresponsible AI any more than there is a responsible or irresponsible watch? Your watch may not keep time accurately, but it is still the watch wearer’s responsibility to arrive to appointments on time. If your friend arrives half an hour late, and blames their watch, or the red lights on the road, will you be annoyed with the watch or the red lights? Or with your friend?   Because ethics vary from person to person, the codification of these ethics into laws are really the only way to govern AI. Currently autonomous vehicles must obey traffic laws, online information systems must obey data protection laws, and autonomous weapons have to comply with the laws of war. Unfortunately, the world of AI is rapidly outpacing the associated laws and the legal world has much catching up to do.   Teaser   Are you wondering what SAS is doing in the fight for Responsible AI? Stay posted for my next blog where I will cover this topic.   Real or Bot?       In a recent Microsoft Teams conversation with SAS employee @michaelerickson, Teams “decided” to make Michael look a bit bot-like.   For More Information   SAS Responsible AI webinar by Ray Greenwood of SAS  Start at minute 10. Bot traffic article Washington Post article by Christian Davenport, December 3, 2017   Interesting Read Klara and the Sun by Kazuo Ishiguro. A thought-provoking story about AI “friends.”   Read Part 2 - AI Ethics Part 2: Trustworthy AI at SAS    Find more articles from SAS Global Enablement and Learning here. ... View more

This blog is based entirely on the demonstration and code written by @TammyJackson , SAS Principal Research Statistician Developer. Special thanks also to Javier Delgado and Thiago Quirino for developing the SAS EXTLANG package and some of the code shown here.   The abundance of real-time and near-real-time data available on the internet has created an amazing opportunity for forecasters to dramatically improve nowcasts. Did you know that data collected in real time can be used to produce high quality nowcasts directly within SAS Visual Forecasting? That’s right, the EXTLANG package lets us pull data from the internet directly into our PROC TSMODEL process.   Background   Tammy Jackson was initially inspired by a presentation at the Women in Statistics and Data Science Conference in 2019 by Samantha Winder, Emilia Lia and Spencer Wood from the University of Washington. In that presentation, Winder et al. compared visitor counts in state and national parks to social media posts including Flickr, Instagram, Twitter. As you can see below, the graphs comparing the seasonal trends from National Park Service surveyed user days (e.g., from trail cams) to the seasonal trends from geolocated Flickr photograph user days correlate very nicely.      Sessions et al 2016 as reported in Winder et al 2019[   Winder et al. successfully came up with a highly accurate model based solely on items they could collect online; it did not use any on site information (trail cams, etc.). Below see the modeled (line) versus observed (dashed line) visitation at an out-of-sample site (Snoqualmie Lake).     Nowcasting with SAS Visual Forecasting Using Real Time Exogenous Variables   For best results, nowcasting with real time exogenous variables, use exogenous (input) variables that are highly correlated with your dependent variable. You will need to be able to obtain current or recent values for the input variables, obtain quality forecasts of the input variables, and collect the input variable data in real time.   Let’s follow Tammy Jackson’s example and see how we can accomplish this!   Tammy Jackson’s Beer Sales Examples     Tammy began with beer sales data from DurstExpress in Germany (Pan, W.H. 2022). She acquired daily data for 2019.   SECTION 1: USING FORECASTED AIR TEMPERATURE VERSUS ACTUAL AIR TEMPERATURE IN THE FORECAST PERIOD Three exogenous factors are included in the data: hours of daylight public holiday (binomial variable where 0 = no or 1 = yes) daily mean air temperature in °C   In initially exploring the data of beer sales in Germany, we see that hours of daylight and mean air temperature are highly correlated with the number of crates of beer sold.     These data are shown graphed below, where the: Blue circles are the actual data (crates of beer sold each date) Smooth black sinusoidal curve is the hours of daylight Jagged red line is the mean temperature Blue line at the bottom with nine peaks indicates public holidays Note that Tammy scaled these to view them together.     By visually looking at the data we can zoom in on Wave One and Wave Two. Let’s fit a model to the German Beer Data and try several model families:   ARIMA with regression variables Exponential smoothing Unobserved components with regression variables   And let’s see if the exogenous variables are significant. hours of daylight public holiday daily mean air temperature   We will use PROC TSMODEL. We will use rc = dataFrame.AddX() statements to add the exogenous variables and define how to treat them. See the code below:   proc tsmodel data = sascas1.germanbeerdatafull LOGCONTROL= (ERROR = KEEP WARNING = KEEP NOTE=KEEP none=keep) puttolog=yes outlog=sascas1.outlog outobj = ( outEST = sascas1.OUTEST_ind (replace = YES) outfor = sascas1.outfor (replace = YES) outstat = sascas1.outstat (replace = YES) outsel=sascas1.outselect (replace = YES) ) errorstop = YES lead=7 ; id date INTERVAL = day setmissing = MISSING trimid=right ; var crates_sold hours_of_daylight public_holiday mean_temperature ; require atsm; /* Use the automatic time series modeling (ATSM) package */ print outlog; submit; declare object dataFrame(tsdf); declare object diagSpec(diagspec); declare object diagnose(diagnose); declare object inselect(selspec); declare object forecast(foreng); rc = dataFrame.Initialize(); /* Add Variable to be FORECAST */ rc = dataFrame.AddY(crates_sold); /* Add INDEPENDENT exogenous variables to model */ /* All variables added will be considered as input by models that support independent variables */ /* You can add and remove the x variables just by commenting out the AddX method statements */ rc = dataFrame.AddX(hours_of_daylight, 'required', 'NO', 'EXTEND','stochastic'); rc = dataFrame.AddX(public_holiday, 'required', 'NO', 'EXTEND','stochastic'); rc = dataFrame.AddX(mean_temperature, 'required', 'NO', 'EXTEND','stochastic'); /* OPEN THE DIAGNOSE SPEC */ /* Add some ARIMA model info to consider ARIMA family of models */ rc = diagSpec.open(); if rc < 0 then do; stop; end; rc = diagSpec.setTransform('TRANSFORM', 'NONE'); rc = diagSpec.setARIMAX('CRITERION', 'AIC'); rc = diagSpec.setARIMAX('IDENTIFY', 'ARIMA'); /* Add some ESM model info to consider ESM family of models */ rc = diagSpec.setESM('method', 'bestn'); if rc < 0 then do; stop; end; /* Add some UCM model info to consider UCM family of models */ rc = diagSpec.setUCM(); if rc < 0 then do; stop; end; rc = diagSpec.close(); if rc < 0 then do; stop; end; /* Initializing from the dataFrame bring in the input variables */ rc = diagnose.Initialize(dataFrame);if rc < 0 then do; stop; end; /* setSpec brings in the model families */ rc = diagnose.setSpec(diagSpec); if rc < 0 then do; stop; end; rc = diagnose.Run(); if rc < 0 then do; stop; end; ndiag = diagnose.nmodels(); /* Define selSpec object*/ /* Create a selection list */ rc = inselect.Open(ndiag); rc = inselect.AddFrom(diagnose); rc = inselect.close(); /* Define forecast object*/ rc = forecast.Initialize(dataFrame); if rc < 0 then do; stop; end; rc = forecast.setOption('LEAD',7); if rc < 0 then do; stop; end; rc = forecast.setOption('ALPHA', 0.05); if rc < 0 then do; stop; end; rc = forecast.setOption('CRITERION','AIC'); if rc < 0 then do; stop; end; /* Add the selection list to the forecast engine */ rc = forecast.AddFrom(inselect); if rc < 0 then do; stop; end; rc = forecast.run(); if rc < 0 then do; stop; end; /* Collect results */ declare object outEst(outEst); declare object outfor(outfor); declare object outsel(outselect); declare object outstat(outstat); rc = outEst.collect(forecast); rc = outfor.collect(forecast); rc = outstat.collect(forecast); rc = outsel.collect(forecast); endsubmit; run; quit;   Results for the entire year are shown below for the model ARIMA (111)(101)7 NOINT. We see that the p-values for the public holiday and the mean temperature are very low indicating that these are significant exogenous factors. Therefore they are used in the model.     The three models that were fitted are the ARIMAX, the ESM, and the UCM model. We can see in the screen shot below that the ARIMAX model did the best based on the Root Mean Square Error, the Mean Absolute Percent Error and the Akaike Information Criterion.     Below we do see the forecast for the entire year and we see that the fit of the model over the year looks good. We do note annual seasonality, which we would capture if we had more years of data.     Recall the two distinct waves in the data that Tammy identified visually.   Let’s forecast each of those waves. First, let use the entire year of data to identify a model. Then let’s use that model along with historical data (up to the point the wave starts) to forecast each wave. Mean temperature is an input and for one case we had missing values during forecast wave period. An exponential smoothing model will be used to forecast the temperature input. Let’s compare: a) using an ESM forecast on the input (the stochastic forecast) versus b) using the actual temperature data (as a proxy forecast) In reality we would not have the actual data but would have pretty good National Weather Forecast at least one day in advance, so we are using the actual temperature data as a proxy for that.   First Wave Forecast   In the screen capture below, the red line is the forecast of number of crates of beer sold using the forecasted temperature input (the stochastic forecast). The blue line is the forecast of the number of crates of beer sold using the actual temperature data (as a proxy for the forecast of the independent variable temperature). As anticipated, we see that using the actual temperature data as an input improves our forecast.   We see this visually.     And we confirm this by comparing the MAPE using the stochastic temperature forecast (perr_NF) versus using the actual temperature (perr_F), and we see that the forecast that used the actual temperature as in input during the forecast period is better.     Second Wave Forecast   We repeat this on the second wave of data. Again, we see that using the actual temperature data as an input improves our forecast.             SECTION 2: PULLING IN ONLINE DATA WITHIN PROC TSMODEL   In the above example it would have been useful to have recent high quality forecast of the temperature data to facilitate nowcasting.   Now that we are convinced using online real time data can improve our forecasts, how do we pull it in to SAS Visual Forecasting in real time and integrate the real time data into our forecast?   We can do it using SAS’s EXTLANG package for python developed by Javier Delgado and Thiago Quirino at SAS. Let’s illustrate this using historic weather data available from the US National Weather Service’s National Centers for Environmental Information (NCEI). The screen capture below shows the web interface for that NWS data, however we will be pulling this in using python and the EXTLANG package.     There are a few pre-steps: 1. Download pyncei 1.0 and install in Python. You can get a token for access here . 2. A python package named pytrends works with Google trends to download weather data via the internet. We use it to download data from the US National Oceanic and Atmospheric Administration (NOAA). We must install pytrends prior to creating our SAS program, because we cannot have a pip install statement within PROC TSMODEL. See below using a Jupyter notebook with the statement pip install pytrends.     3. Ensure that your administrator has enabled the SAS EXTLANG package in your environment. See more details at the end of this blog and in the documentation for more information. So now we can use python WITHIN PROC TSMODEL to pull in internet data! The following code illustrates this by pulling in US National Oceanic and Administration (NOAA) data using a package from NOAA. In this example we pull in the data as a csv and then import it. But we could also pull in the data using arrays, as we will demonstrate later on in the Search Term example. proc tsmodel data = sascas1.obs1 outobj = ( pylog=sascas1.pylog (replace = YES) ) LOGCONTROL= (ERROR = KEEP WARNING = KEEP NOTE=KEEP none=keep) puttolog=yes outlog=sascas1.outlog errorstop = YES lead = 11 id date INTERVAL = day ; var crates_sold; /* Use the External Language (EXTLANG) package as well as the ATSM package */ require atsm extlang; print outlog; submit; declare object py(PYTHON3); declare object pylog(OUTEXTLOG); rc = py.Initialize(); rc = py.PushCodeLine('from noaa_sdk import NOAA'); rc = py.PushCodeLine('from numpy'); rc = py.PushCodeLine('from array import *'); rc = py.PushCodeLine('import pandas as pd'); rc = py.PushCodeLine('import numpy as np'); rc = py.PushCodeLine("keywords = ['startTime', 'temperature']"); rc = py.PushCodeLine("row=[]"); rc = py.PushCodeLine("n = NOAA()"); rc = py.PushCodeLine("res = n.get_forecasts('27615', 'US')"); rc = py.PushCodeLine("a = [ ['startTime', 'temperature'] ]"); rc = py.PushCodeLine("j=1"); rc = py.PushCodeLine("for i in res:"); rc = py.PushCodeLine(" row = [i[keywords[0]],i[keywords[1]]]"); rc = py.PushCodeLine(" a.insert(j, row)"); rc = py.PushCodeLine(" j=j+1"); rc = py.PushCodeLine("pd.DataFrame(a).to_csv("V:\<path>\temperaturefcsts.csv",index=False)"); /* Run */ rc = py.Run(); rc = pylog.Collect(py,'EXECUTION'); ENDSUBMIT; RUN; title "Import Hourly Temperature Forecasts for Zipcode 27615"; proc import out= datasets.nws_temperature_fcst(rename=(VAR1=timeid VAR2=temperature_fcst)) REPLACE datafile="V:\presentations\FFC2022\tempfcsts.csv" DBMS="csv"; getnames=no; datarow=3; run; data datasets.nws_temperature_fcst; set datasets.nws_temperature_fcst; format timeid DATETIME.; run; proc print data=datasets.nws_temperature_fcst; run;   And we have imported hourly observations of NOAA temperature forecast data.     If we want to accumulate those data to, for example, daily max, we could easily also do that in PROC TSMODEL.   /* Accumuate hourly forecasts to daily (max) temperature */ proc tsmodel data = sascas1.nws_temperature_fcst LOGCONTROL= (ERROR = KEEP WARNING = KEEP NOTE=KEEP none=keep) puttolog=yes outlog=sascas1.outlog outarray = sascas1.dailyTempFcst errorstop = YES ; id timeid INTERVAL = DTDAY setmissing = MISSING trimid=right ; var temperature_fcst / acc=max; require atsm; print outlog; submit; declare object dataFrame(tsdf); endsubmit; run; quit;   With results such as those shown below.     SECTION 3: USING GOOGLE SEARCH TERM RECORDS AS EXOGENOUS INPUTS TO FORECAST BEER SALES WITHIN PROC TSMODEL   We will conduct three separate forecasts (nowcasts) within PROC TSMODEL. Once they are run we will compare the accuracies. In each forecast we will use different inputs to determine the amount of beer sold. For inputs we will use: 1. For forecast 1: Default forecast of mean temperature as inputs in the forecast period plus the search term magnitude also as inputs 2. For forecast 2: Actual temperatures as inputs in the forecast period plus the search term magnitude also as inputs 3. For forecast 3: Search term magnitude are the only inputs in the forecast period Let’s describe each of these in a little more detail.   The Google Trends Search Term data are based on the daily number of searches for each term. We see below data for daily searches for: “weather” (blue) “beer” (orange) “temperature (yellow) “hot” (green) Rather than being represented as the actual number of searches, the Google Trends normalizes the data to a scale of 0 to 100 with 100 being the maximum number of daily searches for an individual term in the range of the data. So for example, we see in the Google trends data graph below that the highest number of daily searches was for weather and it occured in July. This date receives a value of 100 for weather searches and all other data are relative to that range maximum.   1) DEFAULT FORECAST OF MEAN TEMPERATURE is used as an input for the forecast period in addition to the search term magnitudes. The default forecast (mean_temperature_NF) uses an ESM model to forecast the mean temperature during the forecast period. We also include search term data. The errors for this model are perr_NF. The results are collected as outfor in the code provided. Inputs available to Forecast 1 are: hours_of_daylight public_holiday mean_temperature_NF (modeled with ESM) search term data: BEER HEAT TEMPERATURE WEATHER 2) ACTUAL TEMPERATURE VALUES are used as the inputs for the forecast period in addition to the search term magnitudes. The errors for this are perr_F. The results are collected as outfor2 in the code provided. (Note: In real life you wouldn’t be able to use actual values for the current day, but you could use actual values up to the current day and a good weather forecast from the NWS for the current day.) Inputs available to Forecast 2 are: hours_of_daylight public_holiday mean_temperature (actual temperature values) search term data: BEER HEAT TEMPERATURE WEATHER 3) INPUTS ARE SEARCH TERM DATA ONLY In this case the forecast with mean_temperature is removed entirely, and ONLY the search term data for beer, heat, temperature, weather (normalized distributions) are used as inputs. The errors for this are perr_ST (where ST stands for search term.) The results are collected as outfor3. Inputs available to Forecast 3 are: public_holiday search term data: BEER HEAT TEMPERATURE WEATHER mean_temperature not available to the forecast model hours_of_daylight not available to the forecast model When you get down to the results, you will be amazed to see that just using the search terms (forecast 3) gives a forecast just as good as when we use the ESM to forecast the mean temperature (forecast 1); in fact, it’s slightly better!   Keep in mind that the quality of the independent variables in the data makes a difference in the results during the forecast period.   Let’s get started: In your environment you will need to open a CAS session, identify your port, username, password, et cetera. I have not included code for that. If you need help with that starting code see the documentation.   Once we have opened a CAS session, we need libname statements to access and write data tables.   We use a DATA STEP to pull in the time period we are interested in. Note that we must ensure that the input data set is the size of the desired output data set. It’s best if your input data set covers the same time period as the real time data that we pull in using python.   Within the DATA STEP, let’s create time periods of interest (wave1 and wave2).   We start our PROC TSMODEL and set up an outarray for all of our variables as shown in the screen capture below. We also create a pylog so that we can access log information from our python program after it runs. And we create some forecasting objects, typical of any forecasting project: parameter estimates outest outest2 outest3 forecasted values outfor outfor2 outfor3 model statistics outstat outstat2 outstat3 selection statistics outselect outselect2 outselect3 Recall that we must ensure that the input data set is the size of the desired output data set. Below see that lead = 0 so that the size of the output data set doesn’t grow and therefore is the same as the size of the input data set.   As you see in the OUTARRAY statement above, our variables are: ID YEAR MONTH DAY WEATHER TEMPERATURE BEER HEAT crates_sold_NF mean_temperature_NF During the forecast period actual values were removed and replaced with missing values for crates_sold_NF and mean_temperature_NF.     Now, we need to require two packages, as shown in the screen shot below: ATSM – automatic time series modeling package EXTLANG – external languages package, which will allow us to use python programming language within our PROC TSMODEL code! For more information on the EXTLANG package see the section on this at the end of this blog and see the documentation here and here. With the EXTLANG package, we are now free to submit our user-defined code (both SAS code and Python code!) between the submit and endsubmit statements as shown in the screen capture below.   The first thing we do inside of our submit and endsubmit statements is to declare our python object and our python log, as shown in the screen capture above:   declare object py(PYTHON3); declare object pylog(OUTEXTLOG);   This ensures that we can see a log in case of errors. We use return code statements to run the Python lines of code that we need. First let’s initialize python. Then we add the variables we need (to make them available to our python code), such as   rc = py.AddVariable(YEAR, ‘READONLY’, ‘FALSE’);     Now we can submit the python code lines using PushCodeLine statements as shown in the screen capture below. We start by pulling in the python packages we need. Remember that we need to import the pytrends API so that we can pull in the NOAA data from the internet in real time.   Note that the python code is inside the quotes. If you have quotes within quotes, you need to make sure that either:   the exterior quotes are double quotes and the interior quotes are single quotes, or the exterior quotes are single quotes and the interior quotes are double quotes.   For example:   rc = py.PushCodeLine(“pytrends = TrendReq(h1=’en-US’, tz=0, retries=10)”);   In the code snippet below we see the line where this is illustrated:     We continue to use rc = py.PushCodeLine to select and sort keywords, select the geography (DE for Deutschland, which is Germany). We also set up a wait of 6 seconds.     In the screen capture below we prepare the containers and start loops to retrieve data using pytrends. We set up the data dictionary with the python code trends = dict.fromkeys(geo). We set up an empty error list with the code errors_list = [] which will populate if there are errors. We loop over the geographies and the keywords. Notice that we also include the python code time.sleep(wait) (set at 6 seconds in the screen capture above) to help avoid timeout errors.     We will initialize the arrays to receive the data. Recall that we must ensure that the input data set is the size of the desired output data set. See below we use (212) to ensure that the arrays are the same size as the input and output data sets.     Next we transfer the pytrends search term data to local arrays.     And we run the python program and collect the python log information.   Now we create the date time stamp for the imported data to ensure that the internet data that we have pulled in using pytrends are properly aligned with the time series data we are forecasting (beer sales data).     Example output for the first four observations of the search term data is shown below.     Running the forecasts:   As in any PROC TS MODEL code we declare our objects including the FORENG object for our first forecast.     We initialize our dataFrame and add our forecast variable, which is the number of crates sold crates_sold_NF. And here we extend using a stochastic model (an exponential smoothing model).     We add some ARIMA model information including that we will use the Akaike Information Criterion as the model selection criterion.   We also add ESM and UCM models for consideration.   Initializing from the dataFrame brings in the input variables. Then we create a selection list. We define our forecast object, set lead time using a macro variable, set alpha to 0.05, and set the selection criterion to Akaike Information Criterion. We will also add the selection list to the forecast engine.     And, as with all PROC TSMODEL code, we collect our results. Here ends the first forecast.     We repeat similar code for forecast 2.       Repeat for forecast 3.     And finally our endsubmit statement, which ends all of our user-defined code.   We use some DATA STEP code to create three data sets with the information from our three forecasts: • forecast1 • forecast2 (forecast with actual inputs from the internet) • forecast3 (forecast with stochastically forecasted inputs) We can then print and compare the results.     Results   We merge our three forecast tables and create variables for us to look at: • perr_NF – the percent error for the model built with the ESM-modeled mean_temperature as an input • perr_F – the percent error for the proxy model (forecast with actual mean_temperature data from the internet) • perr_ST – the percent error for the model that uses only the search term data as inputs     Plot out stochastic and proxy forecast results.       Compare the means of the errors.   Evaluating the errors using PROC MEANS, we see that: • perr_NF is the worst. Recall that perr_NF is the percent error for the model built with the ESM-modeled mean_temperature as an input • perr_F is the best. Recall that perr_F is the percent error for the proxy model (forecast with actual mean_temperature data from the internet) • perr_ST is slightly better than perr_NF! Recall that perr_ST is the percent error for the model that uses ONLY the search term data as inputs. We find that even though the search terms didn’t fit as well when modeled together with the other inputs, they were slightly better predictors when used alone.     Conclusions Data available in real time can be successfully incorporated into models to produce quality forecasts. The quality of the forecast of the exogenous variables can have a major impact on the quality of the forecasts of the dependent variable. Free data are available, such as Historical weather – US National Centers for Environmental Information (NCEI) Current weather forecasts – US National Weather Service (NWS) Internet search term data – Google Trends All of the above data can be accessed via Python within PROC TSMODEL Integrating Python code into the forecasting process is necessary to produce a quality forecast in real time Summary We see that we were able to get good nowcasting results by incorporating data from the internet. We did this directly in SAS Visual Forecasting using PROC TSMODEL and the EXTLANG package that allows us to include python code in our user-defined code. Again, let me reiterate. We are able to bring in internet data and use it directly in SAS Visual Forecasting within PROC TSMODEL. This is a game changer. A huge thank you to Tammy Jackson, SAS Principal Research Statistician Developer, for demonstrating how this is possible. Thank you also to Javier Delgado and Thiago Quirino for developing the EXTLANG package that makes this possible. Full Code to run the 3 forecasts :     /*********************************** Beth Open CAS Code **************************************/ /* REPLACE THIS WITH YOUR OWN STARTING CODE */ cas BethSession sessopts=(caslib=casuser timeout=1800 locale="en_US"); *libname ANALYTIC cas caslib = “ANALYTIC”; *libname VF cas caslib="VF"; libname CASUSER cas caslib="CASUSER"; *cas Sessionxxx terminate; Data PUBLIC.germanbeerfull (drop = ST_weather_O ST_temperature_O ST_beer ST_temperature ST_weather); set PUBLIC.GERMANBEERDATA_ST; proc print; run; /*********************************************************************************************/ /* German Beer Data January 1, 2019 to July 31, 2019 */ data PUBLIC.JanJul2019; set PUBLIC.germanbeerdata_ST(where=(date GE '01JAN2019'D and date LE '31JUL2019'D)); run; /* Some time periods of interest */ %let wave1 = '01JUN2019'D; %let wave1end = '30JUN2019'D; %let lead1 = 29; %let wave2 = '19JUL2019'D; %let wave2end = '31JUL2019'D; %let lead2 = 12; %let table = PUBLIC.JanJul2019; %let start = &wave2; %let wave = &wave2; %let lead = &lead2; /*********************************************************************************************/ /* PROC TSMODEL STARTS HERE */ /*********************************************************************************************/ proc tsmodel data = PUBLIC.JanJul2019 OUTARRAY = PUBLIC.outarray outobj = ( pylog = PUBLIC.pylog (replace = YES) outEST = PUBLIC.OUTEST_ind (replace = YES) outfor = PUBLIC.outfor (replace = YES) outstat = PUBLIC.outstat (replace = YES) outsel = PUBLIC.outselect (replace = YES) outfor2 = PUBLIC.outfor2 (replace = YES) outEST2 = PUBLIC.OUTEST_ind2 (replace = YES) outstat2 = PUBLIC.outstat2 (replace = YES) outsel2 = PUBLIC.outselect2 (replace = YES) outfor3 = PUBLIC.outfor3 (replace = YES) outEST3 = PUBLIC.OUTEST_ind3 (replace = YES) outstat3 = PUBLIC.outstat3 (replace = YES) outsel3 = PUBLIC.outselect3 (replace = YES) ) LOGCONTROL= (ERROR = KEEP WARNING = KEEP NOTE=KEEP none=keep) puttolog=yes outlog=PUBLIC.outlog errorstop = YES lead=0 ; id date INTERVAL = day; var crates_sold hours_of_daylight public_holiday mean_temperature ; OUTARRAY ID YEAR MONTH DAY WEATHER TEMPERATURE BEER HEAT crates_sold_NF mean_temperature_NF; require atsm extlang; print outlog; submit; declare object py(PYTHON3); declare object pylog(OUTEXTLOG); rc = py.Initialize(); /* This install needs to be done prior to running SAS code */ *rc = py.PushCodeLine('!pip install pytrends'); RC = PY.AddVariable( YEAR, 'READONLY', 'FALSE'); RC = PY.AddVariable( MONTH, 'READONLY', 'FALSE'); RC = PY.AddVariable( DAY, 'READONLY', 'FALSE'); RC = PY.AddVariable(BEER, 'READONLY', 'FALSE'); RC = PY.AddVariable(HEAT, 'READONLY', 'FALSE'); RC = PY.AddVariable(TEMPERATURE, 'READONLY', 'FALSE'); RC = PY.AddVariable(WEATHER, 'READONLY', 'FALSE'); rc = py.PushCodeLine('import time'); rc = py.PushCodeLine('import pandas as pd'); rc = py.PushCodeLine('import numpy as np'); *BETH ADDED THIS LINE; rc = py.PushCodeLine('import pytrends'); rc = py.PushCodeLine('from datetime import datetime'); rc = py.PushCodeLine("from pytrends.request import TrendReq"); rc = py.PushCodeLine("pytrends = TrendReq(hl='en-US', tz=0, retries=10)"); /* keywords */ rc = py.PushCodeLine("keywords = ['weather', 'temperature', 'beer', 'heat']"); /* Sort the list */ rc = py.PushCodeLine('keywords.sort()'); /* locations */ rc = py.PushCodeLine("geo = ['DE']"); rc = py.PushCodeLine("geo.sort()"); /* Perform searches */ /* wait in seconds */ rc = py.PushCodeLine("wait = 6"); /*print('Number of queries to do: ', len(keywords) * len(geo)) */ /*# Prepare containers */ rc = py.PushCodeLine("trends = dict.fromkeys(geo)"); rc = py.PushCodeLine("errors_list = []"); rc = py.PushCodeLine("cnt = 1"); /* Start loops to retrieve data from pytrends */ rc = py.PushCodeLine("for g in geo:"); rc = py.PushCodeLine(" trends[g] = {}"); rc = py.PushCodeLine(" for k in keywords:"); rc = py.PushCodeLine(" try:"); rc = py.PushCodeLine(" time.sleep(wait)"); rc = py.PushCodeLine(" pytrends.build_payload([k], timeframe='2019-01-01 2019-07-31', geo=g, gprop='')"); rc = py.PushCodeLine(" trends[g][k] = pytrends.interest_over_time()[k]"); rc = py.PushCodeLine(" except:"); /*print(cnt, ') Error: ', g, ' & ', k)*/ rc = py.PushCodeLine(" errors_list.append([g,k])"); rc = py.PushCodeLine(" cnt+=1"); /*print('\nDone -', len(errors_list), 'errors left')*/ /* Initialize the arrays to receive data */ rc = py.PushCodeLine("ID = np.zeros(212)"); *BETH ADDED THIS LINE; rc = py.PushCodeLine("YEAR = np.zeros(212)"); rc = py.PushCodeLine("MONTH = np.zeros(212)"); rc = py.PushCodeLine("DAY = np.zeros(212)"); rc = py.PushCodeLine("BEER = np.zeros(212)"); rc = py.PushCodeLine("HEAT = np.zeros(212)"); rc = py.PushCodeLine("TEMPERATURE = np.zeros(212)"); rc = py.PushCodeLine("WEATHER = np.zeros(212)"); /* Transfer the pytrends data to local arrays */ rc = py.PushCodeLine("for g in geo:"); rc = py.PushCodeLine(" for k in keywords:"); rc = py.PushCodeLine(" keys = trends[g][k].keys()"); rc = py.PushCodeLine(" i=0"); rc = py.PushCodeLine(" for dk in keys:"); rc = py.PushCodeLine(" if k == 'weather':"); *rc = py.PushCodeLine(" ID[i] = datetime.timestamp(dk)"); rc = py.PushCodeLine(" YEAR[i] = dk.year"); rc = py.PushCodeLine(" MONTH[i] = dk.month"); rc = py.PushCodeLine(" DAY[i] = dk.day"); rc = py.PushCodeLine(" WEATHER[i] = trends[g][k][dk]"); rc = py.PushCodeLine(" if k == 'beer':"); rc = py.PushCodeLine(" BEER[i] = trends[g][k][dk]"); rc = py.PushCodeLine(" if k == 'temperature':"); rc = py.PushCodeLine(" TEMPERATURE[i] = trends[g][k][dk]"); rc = py.PushCodeLine(" if k == 'heat':"); rc = py.PushCodeLine(" HEAT[i] = trends[g][k][dk]"); rc = py.PushCodeLine(" i = i+1"); /* Run */ rc = py.Run(); rc = pylog.Collect(py,'EXECUTION'); /* Create the date timestamp for the imported data. This is not necessary for processing if the pytrends data is aligned with the time series data. However, this is a check to make sure that the data is properly aligned. */ do i=1 to _LENGTH_; ID[i] = MDY(MONTH[i],DAY[i],YEAR[i]); end; /* Withhold some data during the forecast period */ do i=1 to _LENGTH_; crates_sold_NF[i] = crates_sold[i]; if ( date[i] GT &wave ) then crates_sold_NF[i] = .; mean_temperature_NF[i] = mean_temperature[i]; if ( date[i] GT &wave ) then mean_temperature_NF[i] = .; end; declare object dataFrame(tsdf); declare object diagSpec(diagspec); declare object diagnose(diagnose); declare object inselect(selspec); /**********************************************************************************************************/ /* CODE FOR FIRST FORECAST: Uses default forecast (ESM) of temperatures as inputs in the forecast period */ /**********************************************************************************************************/ declare object forecast(foreng); rc = dataFrame.Initialize(); /* Add Variable to be FORECAST - data is missing during forecast period */ rc = dataFrame.AddY(crates_sold_NF); /* Add INDEPENDENT Variables to model */ /* All variables added will be considered as input by models that support independent variables */ /* You can add and remove the x variables just by commenting out the AddX method statements */ rc = dataFrame.AddX(hours_of_daylight, 'required', 'NO', 'EXTEND','stochastic'); rc = dataFrame.AddX(public_holiday, 'required', 'NO', 'EXTEND','stochastic'); /* NOTE - Missing during forecast period, data will be extended using ESM model */ rc = dataFrame.AddX(mean_temperature_NF, 'required', 'NO', 'EXTEND','stochastic'); /* Search Term data directly from pytrends */ rc = dataFrame.AddX(BEER, 'required', 'NO', 'EXTEND','stochastic'); rc = dataFrame.AddX(HEAT, 'required', 'NO', 'EXTEND','stochastic'); rc = dataFrame.AddX(TEMPERATURE, 'required', 'NO', 'EXTEND','stochastic'); rc = dataFrame.AddX(WEATHER, 'required', 'NO', 'EXTEND','stochastic'); /* OPEN THE DIAGNOSE SPEC */ /* Add some ARIMA model info to consider ARIMA family of models */ rc = diagSpec.open(); if rc < 0 then do; stop; end; rc = diagSpec.setTransform('TRANSFORM', 'NONE'); rc = diagSpec.setARIMAX('CRITERION', 'AIC'); rc = diagSpec.setARIMAX('IDENTIFY', 'ARIMA'); /* Add some ESM model info to consider ESM family of models */ rc = diagSpec.setEsm('method', 'bestn');if rc < 0 then do; stop; end; /* Add some UCM model info to consider UCM family of models */ rc = diagSpec.setUCM();if rc < 0 then do; stop; end; rc = diagSpec.close(); if rc < 0 then do; stop; end; /* Initializing from the dataFrame brings in the input variables */ rc = diagnose.Initialize(dataFrame);if rc < 0 then do; stop; end; /* setSpec brings in the model families */ rc = diagnose.setSpec(diagSpec);if rc < 0 then do; stop; end; rc = diagnose.Run();if rc < 0 then do; stop; end; ndiag = diagnose.nmodels(); /* Define selSpec object*/ /* Create a selection list */ rc = inselect.Open(ndiag); rc = inselect.AddFrom(diagnose); rc = inselect.close(); /* Define forecast object*/ rc = forecast.Initialize(dataFrame); if rc < 0 then do; stop; end; rc = forecast.setOption('LEAD',&lead); if rc < 0 then do; stop; end; rc = forecast.setOption('ALPHA', 0.05); if rc < 0 then do; stop; end; rc = forecast.setOption('CRITERION','AIC'); if rc < 0 then do; stop; end; /* Add the selection list to the forecast engine */ rc = forecast.AddFrom(inselect); if rc < 0 then do; stop; end; rc = forecast.run(); if rc < 0 then do; stop; end; /* Collect results */ declare object outEst(outEst); declare object outfor(outfor); declare object outsel(outselect); declare object outstat(outstat); rc = outEst.collect(forecast); rc = outfor.collect(forecast); rc = outstat.collect(forecast); rc = outsel.collect(forecast); put "end first forecast" rc=; /****************************************************************************************/ /* CODE FOR SECOND FORECAST: Uses actual temperatures as inputs in the forecast period */ /****************************************************************************************/ declare object forecast2(foreng); rc = dataFrame.Initialize(); rc = dataFrame.AddY(crates_sold_NF); /* Add INDEPENDENT Variables to model */ rc = dataFrame.AddX(public_holiday, 'required', 'NO', 'EXTEND','stochastic'); /* Note - already has forecast values */ rc = dataFrame.AddX(mean_temperature, 'required', 'NO', 'EXTEND','stochastic'); /* Search Term data directly from pytrends */ rc = dataFrame.AddX(BEER, 'required', 'NO', 'EXTEND','stochastic'); rc = dataFrame.AddX(HEAT, 'required', 'NO', 'EXTEND','stochastic'); rc = dataFrame.AddX(TEMPERATURE, 'required', 'NO', 'EXTEND','stochastic'); rc = dataFrame.AddX(WEATHER, 'required', 'NO', 'EXTEND','stochastic'); /* Use the same model families from above (diagSpec) */ rc = diagnose.Initialize(dataFrame);if rc < 0 then do; stop; end; rc = diagnose.setSpec(diagSpec);if rc < 0 then do; stop; end; rc = diagnose.Run();if rc < 0 then do; stop; end; ndiag = diagnose.nmodels(); /* Define selSpec object*/ rc = inselect.Open(ndiag); rc = inselect.AddFrom(diagnose); rc = inselect.close(); /* Define forecast object*/ rc = forecast2.Initialize(dataFrame); if rc < 0 then do; stop; end; rc = forecast2.setOption('LEAD',&lead); if rc < 0 then do; stop; end; rc = forecast2.setOption('ALPHA', 0.05); rc = forecast2.setOption('CRITERION','AIC'); rc = forecast2.AddFrom(inselect); if rc < 0 then do; stop; end; rc = forecast2.run(); if rc < 0 then do; stop; end; declare object outfor2(outfor); declare object outEst2(outEst); declare object outsel2(outselect); declare object outstat2(outstat); rc = outEst2.collect(forecast2); rc = outfor2.collect(forecast2); rc = outstat2.collect(forecast2); rc = outsel2.collect(forecast2); put "end second forecast" rc=; /*************************************************************************************************/ /* CODE FOR THIRD FORECAST: Uses search term data distributions as inputs in the forecast period */ /*************************************************************************************************/ declare object forecast3(foreng); rc = dataFrame.Initialize(); rc = dataFrame.AddY(crates_sold_NF); /* Add INDEPENDENT Variables to model */ rc = dataFrame.AddX(public_holiday, 'required', 'NO', 'EXTEND','stochastic'); /* Note - already has forecast values */ *rc = dataFrame.AddX(mean_temperature, 'required', 'NO', 'EXTEND','stochastic'); /* Search Term data directly from pytrends */ rc = dataFrame.AddX(BEER, 'required', 'NO', 'EXTEND','stochastic'); rc = dataFrame.AddX(HEAT, 'required', 'NO', 'EXTEND','stochastic'); rc = dataFrame.AddX(TEMPERATURE, 'required', 'NO', 'EXTEND','stochastic'); rc = dataFrame.AddX(WEATHER, 'required', 'NO', 'EXTEND','stochastic'); /* Use the same model families from above (diagSpec) */ rc = diagnose.Initialize(dataFrame);if rc < 0 then do; stop; end; rc = diagnose.setSpec(diagSpec);if rc < 0 then do; stop; end; rc = diagnose.Run();if rc < 0 then do; stop; end; ndiag = diagnose.nmodels(); /* Define selSpec object*/ rc = inselect.Open(ndiag); rc = inselect.AddFrom(diagnose); rc = inselect.close(); /* Define forecast object*/ rc = forecast3.Initialize(dataFrame); if rc < 0 then do; stop; end; rc = forecast3.setOption('LEAD',&lead); if rc < 0 then do; stop; end; rc = forecast3.setOption('ALPHA', 0.05); rc = forecast3.setOption('CRITERION','AIC'); rc = forecast3.AddFrom(inselect); if rc < 0 then do; stop; end; rc = forecast3.run(); if rc < 0 then do; stop; end; declare object outfor3(outfor); declare object outEst3(outEst); declare object outsel3(outselect); declare object outstat3(outstat); rc = outEst3.collect(forecast3); rc = outfor3.collect(forecast3); rc = outstat3.collect(forecast3); rc = outsel3.collect(forecast3); put "end third forecast" rc=; ENDSUBMIT; RUN; /*********************************************/ /* END OF USER-DEFINED CODE FOR PROC TSMODEL */ /*********************************************/ data for(rename=(actual=actual_NF predict=predict_NF error=error_NF std=std_NF upper=upper_NF lower=lower_NF)); set PUBLIC.outfor; run; proc sort data=for; by date;run; *proc print data=for;run; data for2; set PUBLIC.outfor2; run; proc sort data=for2; by date;run; *proc print data=for1;run; data for3(rename=(actual=actual_ST predict=predict_ST error=error_ST std=std_ST upper=upper_ST lower=lower_ST)); set PUBLIC.outfor3; run; proc sort data=for3; by date;run; *proc print data=for1;run; data GermanBeerForecasts; merge &table for for2 for3; by date; err_NF = .; err_F = .; if ( date GT &wave ) then err_NF = ABS(crates_sold - predict_NF); if ( date GT &wave ) then err_F = ABS(crates_sold - predict); if ( date GT &wave ) then err_ST = ABS(crates_sold - predict_ST); perr_NF = err_NF/crates_sold; perr_F = err_F/crates_sold; perr_ST = err_ST/crates_sold; run; proc sgplot data=GermanBeerForecasts; scatter x=date y=crates_sold; series x=date y=predict_NF / lineattrs=(color=red) legendlabel="Stochastic Forecast"; series x=date y=predict / lineattrs=(color=VIGB) legendlabel="Proxy Forecast (Actual)"; series x=date y=predict_ST / lineattrs=(color=green pattern=longdash ) legendlabel="Search Term"; refline &start / axis=x label="Forecast Horizon"; run; title "Search Term is slightly better than using actual data with ESM forecast"; proc means data= GermanBeerForecasts; var perr_NF perr_F perr_ST; run; /* %cas_disconnect; */   MORE DETAILS ON ENABLING THE EXTLANG PACKAGE You will need administrative privileges to enable the EXTLANGE package. In addition: The package requires a Python executable that is specific to the operating system, That is, the SAS administrator must compile/obtain a Python executable and libraries specific to the operating system running SAS. For example, a Unix system requires a Unix executable. You may also need libraries that the Python executable depends on. The executables/libraries need to be stored in a location that is accessible to the SAS system. The location of the libraries/executables should be specified in an xml file in a location that is also accessible to the SAS system. The location of the xml file is specified in the SAS_EXTLANG_SETTINGS environment variable. The xml file: Specifies some permissions Points to python executable and libraries Here is a framework for an xml file similar to what your administrator might use. EXTLANG version “1.0” mode=”ANARCHY” identCheck=”BLOCK” allowALLUsers=”ALLOW”> <DEFAULT scratchDisk=”/tmp” diskAllowList=*/*> <LANGUAGE name=PYTHON2” interpreter=”<path>/python/Python-2.7.16/bin/python2”> <!—some containers don’t have this version of SSL installed, which is the version that the python installer uses --> <ENVIRONMENT name=”LD_LIBRARY_PATH” value=”<path>/openss1-1.0.2k-dist/usr/lib64”> </ENVRIRONMENT> </LANGUAGE> <LANGUAGE name=PYTHON3” interpreter=”<path>/python/rhel7/Python-3.8.8/bin/python3”> <ENVIRONMENT name=”LD_LIBRARY_PATH” value=”<path>/openss1-1.1.1j/lib”> </ENVIRONMENT> </LANGUAGE> <LANGUAGE name=R” interpreter=”<path>/r/R-3.2.5/bin/Rscript”> <ENVIRONMENT name=”LD_LIBRARY_PATH” value=”<path>/r/dependencies/e17/x86_64”> </ENVIRONMENT> </LANGUAGE> </DEFAULT> </EXTLANG> MORE DETAILS ON ENABLING THE EXTLANG PACKAGE You will need administrative privileges to enable the EXTLANGE package. In addition: The package requires a Python executable that is specific to the operating system, That is, the SAS administrator must compile/obtain a Python executable and libraries specific to the operating system running SAS. For example, a Unix system requires a Unix executable. You may also need libraries that the Python executable depends on. The executables/libraries need to be stored in a location that is accessible to the SAS system. The location of the libraries/executables should be specified in an xml file in a location that is also accessible to the SAS system. The location of the xml file is specified in the SAS_EXTLANG_SETTINGS environment variable. The xml file: Specifies some permissions Points to python executable and libraries Here is a framework for an xml file similar to what your administrator might use. EXTLANG version “1.0” mode=”ANARCHY” identCheck=”BLOCK” allowALLUsers=”ALLOW”> <DEFAULT scratchDisk=”/tmp” diskAllowList=*/*> <LANGUAGE name=PYTHON2” interpreter=”<path>/python/Python-2.7.16/bin/python2”> <!—some containers don’t have this version of SSL installed, which is the version that the python installer uses --> <ENVIRONMENT name=”LD_LIBRARY_PATH” value=”<path>/openss1-1.0.2k-dist/usr/lib64”> </ENVRIRONMENT> </LANGUAGE> <LANGUAGE name=PYTHON3” interpreter=”<path>/python/rhel7/Python-3.8.8/bin/python3”> <ENVIRONMENT name=”LD_LIBRARY_PATH” value=”<path>/openss1-1.1.1j/lib”> </ENVIRONMENT> </LANGUAGE> <LANGUAGE name=R” interpreter=”<path>/r/R-3.2.5/bin/Rscript”> <ENVIRONMENT name=”LD_LIBRARY_PATH” value=”<path>/r/dependencies/e17/x86_64”> </ENVIRONMENT> </LANGUAGE> </DEFAULT> </EXTLANG> For more information, see the documentation on External Languages Package here and here. FOR MORE INFORMATION Tammy Jackson’s presentation slides online References Google Trends Data . Kuong, Paulo. (2022, February 19) GitHub - paulokuong/noaa: NOAA Weather Service Python SDK . Mansur, A (2022, May 16) pyneci. National Weather Service. National Weather Service, API Web Service. Pan, W. H. (2022, April 2). The factors of yearly beer sales — Linear model in time series analysis. Medium. Retrieved September 19, 2022. pyncei 1.0 (2022, January 24) SAS Help Center. Veillon. L. (2021, March 2) How to bulk download data from Google Trends using Python  Winder, S., Lia, E., and Wood, S. (2019, October 3-5) Modeling Visitation to Public Lands Using Social Media and Crowd-Sourced Data [Paper presentation]. Women in Statistics and Data Science Bellevue, WA, USA.       ... View more