Incorporating Generative AI models in SAS Visual Forecasting - Part 1 Introduction & Deploying Chronos   This is the first of a two part blog post about pretrained forecasting models and how to integrate them with SAS Visual Forecasting. The second part can be found here.   Introduction to Pretrained Forecasting Models   LLM models such as the GPT models behind ChatGPT are pretrained. In a recent article which was published in Foresight, SAS forecasting experts discussed how will Generative AI influence forecasting software and the challenges that come up in terms of available and domain specific data to develop powerful pretrained models that could outperform traditional techniques in various use-cases. However, pretrained models which are based on neural network architectures, originally designed for natural language processing, are earning ground in the field of forecasting and have shown potential in efficiently predicting time series data across various domains. Even though, we don't expect that those models will replace the traditional forecasting techniques any time soon, pretrained models will keep evolving, becoming more accurate, cheaper and faster as the time goes by. This chapter will explore the fundamentals of pretrained, transformer-based models, their application to forecasting and how to deploy them in SAS Viya.   The Chronos Model   Among the cutting-edge forecasting tools available today, the Chronos model stands out for its performance and versatility. Designed specifically for time series prediction, Chronos leverages the power of transformer architecture to capture complex patterns and dependencies in data. This section will give you a quick introduction to Chronos, but if you want to dive deeper check out the original paper on it here. The great thing about Chronos, which was developed by Amazon Science, is that it is licensed under Apache 2.0, so that we can build on top of it.   The key innovation lies in its time series tokenization process, which maps real-valued observations to a finite set of tokens through scaling and quantization. This allows Chronos to utilize off-the-shelf language model architectures like T5 or GPT-2 with minimal modifications, primarily adjusting the vocabulary size to accommodate the tokenized time series. The model is trained using a standard categorical cross-entropy loss, eschewing distance-aware objectives in favor of allowing the model to learn relationships between nearby tokens implicitly. By treating time series as sequences of tokens without explicit time or frequency information, Chronos demonstrates the general-purpose sequence models in capturing temporal dependencies. The simplicity of this approach, combined with its ability to leverage existing language model training pipelines, makes Chronos a promising framework for developing powerful and adaptable time series forecasting models.   Deploying Chronos as a Container   Now let us get to the meat and potatoes of this post: deploying the Chronos model so that we can integrate with SAS Viya. You can find the full code for this deployment on GitHub.   For the integration the idea is to make Chronos callable via a REST-API endpoint so that we have an easy time of integrating in multiple tools. The API endpoint should have two main properties: 1. Take in the time series data and 2. be able to receive options to tune the model.   A REST-API endpoint is chosen so that we are flexible with how we can call it: Proc HTTP, Proc Python, Open Source Code nodes, SAS Visual Analytics Data Driven Content objects, etc., etc.. In addition creating a containrized workload that can be scaled independant of SAS Viya ensures that your current workloads are not impacted by this additional model.   For that we have written a short Python script which will serve as the point of entry and running the inference on the Chronos model: import torch from chronos import ChronosPipeline from fastapi import FastAPI, HTTPException, Request from pydantic import BaseModel import pandas as pd import os from fastapi.responses import StreamingResponse import io from typing import List, Dict app = FastAPI() class Item(BaseModel): prediction_length: int num_samples: int temperature: float top_k: int top_p: float data: List[float] chronos_model = os.getenv('CHRONOS_MODEL') if chronos_model is None: chronos_model = "amazon/chronos-t5-small" print(f"Running using the {chronos_model} model") pipeline = ChronosPipeline.from_pretrained( chronos_model, ) # Actual prediction process @app.post("/predict/") async def predict(item: Item): try: prediction_length = item.prediction_length num_samples = item.num_samples temperature = item.temperature top_k = item.top_k top_p = item.top_p context = torch.tensor(item.data) except Exception as e: raise HTTPException(status_code=400, detail=f"Error processing input data: {e}") try: forecast = pipeline.predict( context, prediction_length, num_samples=num_samples, temperature=temperature, top_k=top_k, top_p=top_p, ) low, median, high = np.quantile(forecast[0].numpy(), [0.1, 0.5, 0.9], axis=0) dfOut = pd.DataFrame(data={"low": low, "median": median, "high": high}) stream = io.StringIO() dfOut.to_csv(stream, index = False) response = StreamingResponse(iter([stream.getvalue()]), media_type="text/csv" ) response.headers["Content-Disposition"] = "attachment; filename=predictions.csv" return response except Exception as e: raise HTTPException(status_code=500, detail=f"Error during prediction: {e}")   Next we will create a Docker file which we can then run: FROM python:3.12-slim WORKDIR /chronos COPY ./app /chronos/app RUN apt-get update && apt-get install -y --no-install-recommends git RUN pip install git+https://github.com/amazon-science/chronos-forecasting.git RUN pip install pandas fastapi uvicorn pydantic EXPOSE 8000 ARG UID=1001 ARG GID=1001 RUN groupadd -g "${GID}" sas \ && useradd --create-home --no-log-init -u "${UID}" -g "${GID}" sas USER python ENV CHRONOS_MODEL "amazon/chronos-t5-small" CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"] Calling from SAS Studio   Just to test the waters let us quickly build a SAS script to call the Chronos API with a SAS dataset, note that a deeper integration with SAS Visual Forecasting pipelines will be showcased in the next article - here is the script to call it: * Get the Viya Host URL if you deployed chronos here; %let viyaHost=%sysfunc(getoption(SERVICESBASEURL)); * Specify the chronos endpoint; %let chronosEndpoint = /forecast/predict/; filename _ffmIn temp; * Define the structure for the call; data _null_; set sashelp.air(keep=air) end=EoF; file _ffmIn; if _n_ eq 1 then do; predictionLength = '{"prediction_length": ' || put(10, 8.) || ','; num_samples = '"num_samples": ' || put(20, 8.) || ','; temperature = '"temperature": ' || put(1, 8.) || ','; top_k = '"top_k": ' || put(50, 8.) || ','; top_p = '"top_p": ' || put(1, 8.) || ','; dataBase = '"data": ['; put predictionLength; put num_samples; put temperature; put top_k; put top_p; put dataBase; end; if not EoF then do; currentIteration = compress(put(air, 8.) || ','); put currentIteration; end; else if EoF then do; currentIteration = compress(put(air, 8.)); dataClose = ']}'; put currentIteration; put dataClose; end; run; filename _ffmOut temp; * Call the chronos model; proc http method = 'Post' url = "&viyaHost.&chronosEndpoint." in = _ffmIn out = _ffmOut; headers 'Content-Type' = 'application/json' 'Accept' = 'text/csv'; quit; * Import the returned CSV file; proc import file = _ffmOut dbms = csv out = work.results replace; run; quit; * Clean up; filename _ffmIn clear; filename _ffmOut clear; %symdel viyaHost chronosEndpoint; This gives us the following results:   Conclusion and Next Steps   Now to recap we quickly introduced you two pretrained and transformer-based forecasting models and specifically the Chronos model family. Then we took a look at how to deploy it as an API and a container and finally called it in a basic fashion using 'proc http' from SAS Studio. In the next article we will dive deep into the topic of how to integrate Chronos with SAS Visual Forecasting step by step - find it here. ... View more

The State of a report describes the view of a report across sessions. As the report creator you define what the default of a report will be for everyone that opens it up - so make sure that your final save in the editing mode as all of the filters set to the appropriate values and drills states are set as expected. This final state is also known as the Default State of a report, as it is a default if a user opens up the report for the first time.   But as your users use the report they might change the filters and selections as they use it and when they close the report a user specific state for that report is created. This new user specific state will enable the user to pick up from where they left off last time. Note this doesn't change the report itself or the state for any other user, so this is something that happens in the Viewing mode of the report.   If a user wants to go back to the Default State, they can use the report level kebab menu (1) and then select Restore default report state (2) - note we are in the viewing mode of the report:   That every user can pick off of where they last were is a great default and very helpful to a lot of users. But you might disagree in general or for a specific report it doesn't really make sense. So can we change this default behavior? Of course we can.   Let us first talk where this default is set, before we take a look at how to change it:   This rule /reports/reports/*/state (3) grants the right to all Authenticated Users (so not Guests) to Create a report state. What we want to do We want to change this default behavior in two scenarios:  We want to set the default to be prohibited instead of granted We want to deny the ability to create report states for a specific report Now before we get into this there are two things to keep in mind: Users of the SAS Administrator group will not be impacted by this change, but will always be able to create a report state, as long as they clicked Yes when prompted to assume membership of the SAS Administrators group at login If a user has previously created a report state they will still load this report state, it will be overwritten once the report creator makes a change to the underlying report In order to change or create rules there are three main ways of doing it. The first way is by using SAS Environment Manager, which enables us to do this in a visual way by hand, this is great as an introduction to understand the different components and how they relate to each other. The second way is to use the SAS Viya CLI, this is a great way to make use of if you are mass creating rules, changing them in batch or want to document everything. The third and final way are the SAS Viya APIs, these give you the highest degree of flexibility in my opinion, but also require you to write code.   We will be taking a look at all three ways here in order. If you are unfamiliar with how rules work in general within SAS Viya, please read the SAS Environment Manager section, even with you are more interested in an automated way, as I will explain key concepts only there and not repeat them in the other two.   Note: As changing the default behavior for everybody is a one time job we will only be covering it in the section on the SAS Environment Manager. Doing it with the SAS Environment Manager Open up SAS Environment Manager, click on the Rules (1) page, enter /reports/reports/\*/state in the search field (2) & press enter, then select the Conditional Grant row (3) and click on the edit icon (4):   This opens up the Edit Rule dialogue, let us first take a look at it and then discuss all the different elements afterwards: The Object URI is the unique identifier of a piece of content inside of SAS Content. Here you can not only enter a specific URI, but also a URI pattern using an asterixis (*) as a wildcard The Container URI is similar to the Object URI, but it can only be a folder. A rule that is applied to a folder will apply to the folder and all of its members (including subfolders) - Optional The Principal type specifies to whom the rule should be applied. There is five types, lets look at them from the most general to the most specific: Everyone, this means exactly that, everyone with some sort of access to the SAS environment (think of it as Guest + Authenticated Users) Guest, provides access to content of the SAS environment without the need to login - in the screenshot below we see part of the SAS Logon Manager interface where when you click the button Guest (2) you would be signed in as a guest Authenticated Users, this covers everybody that has a SAS user account and is authenticated - in the screenshot below this would happen once you click on the Sign in button (1) Group, this applies the rule to all members of this group, both Groups and Custom groups can be used here User, finally the most granular level is to apply a rule a specific user - as this generates a lot of implementation work it is best to create rules here only if unavoidable The Rule type specifies if this rule Grants some permission or Prohibits it. If you select Prohibit a warning dialogue will pop up with the following message: "A Prohibit rule blocks all access within its scope, even if a more specific Grant rule exists. Are you sure you want to select Prohibit as the rule type?" - so only use Prohibit if it is unavoidable, usually it is better to use more specific Grant rules instead The Condition limits the applicability of the rule, i.e. the rule is only applied if the expression evaluates to true. These conditions are written in the Spring Expression Language (SpEL) and if you want to dive deeper into this topic check out this page in the SAS documentation - Optional The Permissions specify what the rule grants/prohibits. There are seven permissions: Read, Delete, Add, Create, Remove, Secure & Update The Description is meant to help communicate what this rule is for - Optional The Reason is information that is displayed to the user for diagnostic purposes, if they are denied - Optional The Rule status specifies if a rule is active or not You can save any changes made using the Save button Now if we want to change it generally you would have to change the Rule type from Grant to Prohibit.   Ok but now let's talk about the second scenario, where we want to change the report state behavior of a specific report for a specific group of users and for that we will be creating a new rule:   Click on the New icon In the New Rule dialogue click on the folder icon Navigate to the report for which you want to create the rule Click OK 5. Now the current Object URI is /reports/reports/811c3000-d999-4163-b1ec-f976b4e756d7/** which we do not want, as this would change the definition of all the different aspects on the usage of this report so will have to modify it to read /reports/reports/811c3000-d999-4163-b1ec-f976b4e756d7/state 6. Going ahead you can see I changed the Principal type to Group 7. Selected the target group fkt_analyst using the person icon 8. Set the Rule type to Prohibit 9. As the Permissions only select Create, as we want to remove the ability for users to create a report state 10. Add a description to review later on - note I omitted the Reason as there is no failure message to users, that no report state is created 11. Finally click Save   Now when you open up the report as a user of that group change the filters or drill hierarchy, close the report and reopen it the state will remain as the default report state. But if you are a user that isn't part of that group, or has assumed the SAS Administrators group, you will still be able to create a report state. Doing it with the SAS Viya CLI If you are unfamiliar with the SAS Viya CLI, I recommend you check out this excellent tutorial on the SAS Users YouTube channel - watch until 7:02.   For the CLI we will be looking at two ways of doing it: Creating a single rule at a time Creating a single rule but from a file to showcase how you could create many rules at once   The assumption for the following is that you are logged in with the SAS Viya CLI and you have the Authorization plugin installed - more about this specific plugin in the SAS Documentation. It is also assumed that you have already retrieved the reports URI and know the group name to which we will be applying to rule to.   We are going to create a prohibit rule and pass it the report URI, the group name (to be very percise use the group ID, this can be the same as the group name but it can also differ, e.g. group name: SAS Administrators vs. group id: SASAdministrators), the create permission and a description: sas-viya authorization prohibit \ --object-uri /reports/reports/811c3000-d999-4163-b1ec-f976b4e756d7/state \ --group fkt_analyst \ --permissions create \ --description "Prohibits the report state creation for the report Release-Notes" Here is the result of running this command - note that the id here represents the rule id which you can than use via viya-host/authorization/rules/rule-id: { "acceptItemType": "", "acceptType": "", "contentType": "", "createdBy": "gerdaw", "creationTimestamp": "2024-06-07T16:01:11.809Z", "description": "Prohibits the report state creation for the report Release-Notes", "enabled": true, "id": "6a5c2f02-3ea0-499c-acb1-cf5976e5fc6c", "mediaType": "", "modifiedBy": "gerdaw", "modifiedTimestamp": "2024-06-07T16:01:11.809Z", "objectUri": "/reports/reports/811c3000-d999-4163-b1ec-f976b4e756d7/state", "permissions": [ "create" ], "principal": "fkt_analyst", "principalType": "group", "reason": "", "type": "prohibit", "version": 10 } You can delete this rule using the following command: # The id shown below can be retrieved from the output above sas-viya authorization remove-rule \ --id 6a5c2f02-3ea0-499c-acb1-cf5976e5fc6c You can of course also mass add rules using a file - here it is demonstrated with just one and the file that it is stored in is called Report-State-Rule.json: [ { "op": "add", "value": { "description": "Prohibits the report state creation for the report Release-Notes", "objectUri": "/reports/reports/811c3000-d999-4163-b1ec-f976b4e756d7/state", "permissions": [ "Create" ], "principalType": "group", "principal": "fkt_analyst", "type": "prohibit" } } ] Command to create the rule: .\sas-viya.exe authorization create-rules \ --file ./Report-State-Rule.json This returns the following output: "1 rules were created." Doing it with the SAS Viya APIs Using the SAS Viya APIs you can of course call them from many different programming languages. Here I will be demonstrating how to do it using proc http in SAS Studio, this has the advantage of leveraging the build in authentication. But if you are interested in other languages check out the SAS Documentation.   It is also assumed that you have already retrieved the reports URI and know the group name (to be very percise use the group ID, this can be the same as the group name but it can also differ, e.g. group name: SAS Administrators vs. group id: SASAdministrators) to which we will be applying to rule to. Let's get started: * Get the Viya Host URL; %let viyaHost=%sysfunc(getoption(SERVICESBASEURL)); filename in temp; * Create the rule JSON - you could of course also store the file on disk/SAS Content; data _null_; file in; put '{'; put '"type": "prohibit",'; put '"permissions": ["create"],'; put '"principal": "fkt_analyst",'; put '"principalType": "group",'; put '"objectUri": "/reports/reports/811c3000-d999-4163-b1ec-f976b4e756d7/state"'; put '}'; run; filename out temp; * For more information on this API endpoint please check out - https://developers.sas.com/rest-apis/authorization-v8?operation=createRule; proc http url = "&viyaHost./authorization/rules" method = 'Post' out = out in = in oauth_bearer = sas_services; headers 'Accept' = 'application/json'; headers 'Content-Type' = 'application/json'; run; libname out json; title "The new rule was created succesfully - &SYS_PROCHTTP_STATUS_CODE. - &SYS_PROCHTTP_STATUS_PHRASE."; title2 'Navigate to the URI below to find out more'; proc print data = out.links(where=(method='GET') keep=method uri) noobs; quit; title; * Clean up; filename in clear; libname out clear; filename out clear; %symdel viyaHost;   The result should look like this: Supplemental The information in this post are based on the following SAS Support Note. To learn more about the *Rules* page in SAS Environment Manager check out the SAS Documentation. ... View more

Watch this Ask the Expert session to learn how you can use this new editor to optimize coding workflow.    Watch the webinar   You will learn: How to use the SAS Visual Studio code extension. About SAS notebooks. How the integration can help in your everyday work.   The questions from the Q&A segment held at the end of the webinar are listed below and the slides from the webinar are attached.   Q&A When converting your SASNB to a flow, is there a way to name each node through the SASNB? Currently, it just takes the type of program that it is and adds that. I'm not sure if there's already a feature request, but that is a cool suggestion. Please take a look at the issues list and suggest it as a feature.   Can I use this with SAS GRID? SAS Grid does not support IOM or SSH based SAS session creation. You can open a feature request on GitHub for this, to get it assessed further.   Is there any way to export the cells of a notebook into a plain .sas program? Not directly. Currently, there is only the conversion to the SAS flows, where you can generate a SAS file of the complete flow, but not of the separate cells.   What is the biggest advantage (in your opinion) of SAS Visual Studio Code Extension, compared to SAS Enterprise Guide? I like the flexibility of the Visual Studio Code in general because I'm the type of programmer that works with multiple languages. I also prefer the ability to completely customize this experience to my needs. I'm not much of a visual programmer, so I don't need the abilities that Enterprise Guide unlock for me. But I'm very much a code heavy user, so I enjoy the complete customization I can do inside of VS Code.   Is there a way to enable autocompletion as we saw in the presentation, but for custom macros (e.g., Macros Library)? No. You get the same auto complete in the normal SAS coding interface that come as the default. But a great feature idea that you should definitely open up as a feature request on GitHub.   Hi! Clearly, if I already use VS Code this is a great tool. If I'm already using EG or Studio, would there be any benefits to transitioning to VS Code, or should I just continue with my work method? Alternatively, are there any benefits to EG or Studio that I don't get in VS Code? This question is similar to an earlier one, but I’ll address that question at the end. We want to reach you as programmers where you are. A lot of people are in Visual Studio Code and would like to also program SAS there, so that's why this makes a lot of sense. But if you are a person who enjoys maybe just quickly creating a query inside of Enterprise Guide or SAS Studio, then why switch? It's exactly right, you don't have to switch if you are a very code heavy user. Visual Studio Code I believe is an interesting thing to look at. Try it out. It could unlock some new creativity or performance for you. But if you are already very satisfied with Enterprise Guide or SAS Studio, there's no need to switch. You're not missing out on anything.   What is IOM? IOM is a connection technology that is part of SAS 9.4 on a server. I'm not sure what it exactly stands for. It's a connection method to that can be used inside of Enterprise Guide, though most people probably use the metadata-based authentication with Enterprise Guide. IOM is just an alternative to that.   Is it possible to download result data set and report produced from SAS program ran in Visual Studio code? If you have run a SAS code, you can save and copy the results pane, and also the SAS log that was generated in the output window you saw below. That's functionality that comes with VS Code. The SAS extension itself doesn't directly access your local machine; it always works through that VS Code interface. If you generate an output like a PDF that the SAS extension can't display, then the best method on Viya is to save that file to SAS content and just download it from there. For SAS 9.4 servers, you will need to have a mapped shared folder that you can access both on the SAS 9.4 server and your local machine.   How well can you visualize changes made to a SAS notebook tracked in git? The SAS notebook, similar to other notebook conventions, are JSON files that are usually written as a one-line JSON. So, visualizing any changes in there isn't the best of experiences. Even if you add a PewDiePie step to have the broken-up JSON that is more humanly readable, you will still see changes that are related to adding or removing cells. Because cells can also contain output that you save along with it, it's not the greatest experience with any types of these notebooks to save them into git. That's why I'm such a big fan of this conversion to flows as an example to make use of those kind of features.   Is there anything like a 'batch submit' sort of concept, where I submit a program & have my log written to a separate file, so I have a permanent record of what I did? I don't think so because VS Code is very much an interactive application. If you, for example, close it, your SAS session will also be terminated along with that. It's very client dependent in that way. What you could do is if you submitted the log to print, move that log to a separate file, and make use of other functionalities in that way.   Can we use SAS VS code to connect SAS Metadata server in SAS 9.4? And how do I use them to connect to SAS Viya 4? Connecting just to the metadata server is currently not possible. You have to connect to a SAS 9.4 server using either SSH or IOM. Please open a features request accordingly for metadata. Log in on that. The second question on how do you connect to VS Code? Let me quickly jump back to a previous slide (slide number 5) on that to you need to fill out this information. You need to know the Viya endpoint, the compute context that you want to use. A default one is prefilled for you. Also, if you just set up SAS Viya with a relatively recent release then you will already have a client and you can just pace through all those different setup steps. You don't need to have any specialized knowledge for that. This would also enable you to, if you're using DS code, you can open a local SAS program, run it on 9.4, see what the output looks like, connect to your SAS Viya instance, run your code there, and compare and convince yourself that it's still SAS that's running underneath.   In which industries would SAS Visual Code Extension be most useful? The SAS Visual Studio Code Extension is not industry specific; it is a generic SAS coding tool to help you write, read, and run SAS code. The usefulness depends on the user, if you are a code centric person, that does not use any of the visual features of SAS Studio or SAS Enterprise Guide, then try it and see if it works for you.   Does the SAS lib feature come in SAS 9? Currently, as of SAS Visual Studio Code Extension version 1.7.1, this is only supported for SAS Viya. There is already a feature request open to add this feature to SAS 9 based connections, add your support to this feature request by liking it here.   I tried to configure SAS Windows, but I could not see the SAS libraries, but I can write and execute the code. Currently, as of SAS Visual Studio Code Extension version 1.7.1, this is only supported for SAS Viya. There is already a feature request open to add this feature to SAS 9 based connections, add your support to this feature request by liking it here.   Is it possible to connect to SAS Viya 4 from VS Studio using Viya HTTP URL? Which URL will we need to use? It is the same URL that I used to open the SAS Viya 4 environment from your browser.   I don't get the run icon or option. Ensure you installed the SAS Visual Studio Code Extension in Visual Studio Code. Next create an example SAS program, e.g., test.sas, open it and now you should see the run icon in the top right corner.   Is it possible to do the SAS PyCharm extension? There are currently no plans for a SAS extension for PyCharm, as the current effort is on Visual Studio Code.   Can you share the settings.json template that is used in the demo? Yes, you can grab the settings.json template along with additional explanations from this SAS Community article.   SAS Content viewing currently only works with Viya. Are there plans to make it compatible with Local 9.4? Currently, as of SAS Visual Studio Code Extension version 1.7.1, this is only supported for SAS Viya. There is already a feature request open to add this feature to SAS 9 based connections, add your support to this feature request by liking it here.   Are SAS Notebooks compatible with SAS 9? Yes, except the Python cell type, as this requires Proc Python, which is only available with SAS Viya 4.   Is the SAS Integration Client licensed separately from SAS/BASE? No, you can download the SAS Integration Technology Client from this SAS Support page. All you need is an SAS Community profile to sign in.   Does SAS corporation review changes made by external developers to SAS Visual Code Extension to make sure they are appropriate? Yes, there is a code review in place. To read more about this process please check out this document.     Recommended Resources SAS VS Code Extension SAS VS Code Extension GitHub Repository SAS VS Code Extension SAS Community Article Please see additional resources in the attached slide deck.   Want more tips? Be sure to subscribe to the Ask the Expert board to receive follow up Q&A, slides and recordings from other SAS Ask the Expert webinars. ... View more

During my recent Ask the Expert webinar (on-demand link) on the SAS Extension for Visual Studio Code (VS Code) one question came up: Can you please share the example settings.json? The answer is of course yes (and if you are only interested in that, skip to the end), but I thought it is a great opportunity to dive deep into it.    If you are looking for a general article on the SAS Extension for VS Code check out this article.   What is a setting.json and where can I find it?   Configurations inside of VS Code are stored in JSON format. And the settings.json is exactly that store, while there is a global setting.json, if you go to one specific Extension you will get to interact with a settings.json that is reduced. JSON So how do we get there? Go to Manga (cogwheel icon) > Settings > Extensions > SAS > Edit in settings.json - or follow this GIF:   Why would you want to edit the settings.json directly?   That is a fair question, because a lot of the options can be changed through the visual UI that we see just before going to edit the settings.json. And I strongly recommend that you use the visuals for adding settings or profile where you can, as the interfaces are easy to use and have great context information that will help you to better understand the context of each option.   Splitting the settings.json into part.   First we will only focus on the attributes that start with SAS, that is because these settings actually come from the SAS Extension. Depending on what other extensions you have installed you might see many additional attributes in there.   Second will split the SAS attributes into two camps: 1.  SAS.connectionProfiles 2. The rest of the settings, all of them can be edited through visuals. I will not be explaining these here, because as stated above I think they are well explained in the visual part and I would only be reiterating these explanations here.   SAS.connectionProfiles   This section holds all of your connection profiles to different SAS environment. Here we first see the activeProfile which contains the name of the last selected connection profile. After that we see the profiles, which will contain entries for each profile that you have created. Note that as I write this there is currently four types of profiles available SAS Viya, SAS 9.4 (remote - SSH), SAS 9.4 (remote - IOM) and SAS 9.4 (local) - each of the entries will look slightly different depending on the type.   For creating/updating/deleting profiles I recommend that you use the corresponding commands (SAS: Add New Connection Profile, SAS: Update Connection Profile and SAS: Delete Connection Profile). But if you want to add specifc auto exec code or change SAS options within the context of your profile you are correct in going to the settings.json.   Let's assume we have the following:   { "SAS.connectionProfiles": { "activeProfile": "SAS Viya", "profiles": { 1 "Profile Name": { 2 "connectionType": "", 3 "autoExec": [ 4 { "type": "line", "line": "%put Hello World;" }, 5 { "type": "file", "filePath": "/path/to/local/example.sas" }, 6 ... ], 7 "sasOptions": ["pageSize max", ...] } } } } ] }   1. Here you see the name of the profile. 2. I have left the connection type blank for brevity. 3. Now we get to the choice part, we create a new pair that is called autoExec and it contains an array of content. 4. We start a first object to become part of our autoexec. We specified the type as line, which means we have to add an additional key:value pair called line which contains one line of SAS code that will be run as part of our autoexec. 5. Here we have a second object which is of type file and then we have to specify a filePath that refenreces a .sas file on our local machine which will be submitted. 6. Of course you can have as many objects in here as you need, note that they will be run in the order that they are specified here from the first element of the array to the last. 7. You can also add a new pair called sasOptions which takes as an input an array of strings where each string is a SAS option that you would like to set. Seperate each String with a comma and they are also run in order.     The Full Example settings.json   { "SAS.connectionProfiles": { "activeProfile": "SAS Viya", "profiles": { "SAS Viya": { "connectionType": "rest", "endpoint": "https://sasviyaHostURL", "context": "SAS Job Execution compute context", "clientId": "leave empty for the default client (only available on SAS Viya 4)", "clientSecret": "leave empty for the default client (only available on SAS Viya 4)", "autoExec": [ { "type": "line", "line": "%put Hello World;" }, { "type": "file", "filePath": "/path/to/local/example.sas" }, ... ], "sasOptions": ["pageSize max", ...] }, "SAS 9.4 IOM": { "connectionType": "iom", "host": "sasServerAdress", "port": 8591, "username": "yourSASUsername", "sasOptions": [], "autoExec": [] }, "SAS 9.4 Local": { "connectionType": "com", "host": "localhost", "sasOptions": [], "autoExec": [] }, "SAS 9.4 SSH": { "connectionType": "ssh", "host": "sasServerAdress", "saspath": "/path/to/sas/executable", "username": "yourSASUsername", "port": 22, "sasOptions": [], "autoExec": [] } } }, "SAS.results.singlePanel": true, "SAS.flowConversionMode": "Node", "SAS.results.sideBySide": true, "SAS.results.html.enabled": true, "SAS.results.html.style": "(auto)", "SAS.log.showOnExecutionStart": true, "SAS.log.showOnExecutionFinish": true, "SAS.userProvidedCertificates": [ "/path/to/Certificate", ... ] }   ... View more

At the time of writing the SAS Studio Custom Steps repository  contains 57 Custom Steps. Their functionality ranging from importing data, to doing data quality, over synthetic data generation to NLP functionality - quite the impressive spectrum don't you think? Screenshot of the official SAS Studio Custom Step repository on GitHub So with that the question arises how to get them into your SAS Viya environment. You could of course dig through every single folder, grab the .step file and upload it by hand... Ain't nobody got time for that meme So how can we improve this process? We will take a look at three approaches with differing levels of automation: The Automator - fully scripted End-to-End The Lazy - small amount of manual work The Inbetweener (yes this is a cheap joke) The Automator - Automation is Life This first approach makes use of a SAS script that uses the SAS Git functions, macros by the one and only SAS Jedi and SAS Viya APIs. You will have to specify a target folder in which you have already cloned the Custom Steps repository or in which you want it to be cloned and you need to specify a folder in SAS Content that should contain all of the Custom Steps. * Enter the folder in which the SAS Studio Custom Steps are available; %let targetFolderServer = /shared-data/SAS-Studio-Custom-Steps; * If set to one the repository will be cloned into targetFolderServer path; * Set the value to 0 if you already have the repository cloned; %let wantCloning = 1; * If you have the repository already cloned set this to 1 to pull new changes; %let wantPull = 0; * Enter the folder where the SAS Studio Custom Steps should be stored in SAS Content; %let targetFolderContent = /Public/Custom Steps; * Replace previously already existing steps - set to true if you want to replace steps; * Setting this to true is recommend if you pull the repository to get updated steps; %let replaceSteps = false; If you set the macro variable wantCloning to 1, the script will automatically clone the official SAS Studio Custom Step repository from GitHub.   Please note that activating the replaceSteps option can lead to breaking changes and require updates to SAS Studio flows that use updated Custom Steps. This is because Custom Step authors might introduce new required options or other breaking changes.   As you can see, you can also configure this to pull in changes and additions that happen overtime. You can get the full SAS code over at my SAS Snippets GitHub repository - here is a GIF showing this script in action - here I specify a folder into which I have cloned the repository using the Git pane in SAS Studio: Upload and Register all Custom Steps with SAS The Lazy - Click, Zip, Select & Upload This approach was shared with me by @Wilbram-SAS , don't get me wrong I'm not calling him lazy - far from it! The process is simple: go to the GitHub repository  download it as a zip select all of the steps you are interested in upload them to SAS Content   Here is a GIF demonstrating the process: Download GitHub Zip, select the Steps and Upload to SAS Content  The Inbetweener - A little bit of scripting, a little bit of dragging This approach assumes that you have cloned the GitHub repository to your local machine (assuming Windows here) or have downloaded and unzipped it to a folder. We will run a PowerShell command to copy all .step files from the repository folder to one folder so you can quickly drag and drop them into SAS Content. # Navigate to the path cd "C:\Users\<username>\Documents\SAS_Repos\sas-studio-custom-steps" # Create the folder will hold all steps mkdir ..\All-Steps # Copy all steps to the new folder Get-ChildItem -Path ".\*" -Include *.step -Recurse | Copy-Item -Destination "..\All-Steps" Now you can just select the steps that are of interest to you and upload them to SAS Content: Clone, Collect, Upload Conclusion Now you have no excuse not to make use of the awesome utility of all of the Custom Steps available on GitHub - enjoy and let me know which method you end up choosing. I know I'm using the fully automated way of doing things!   ... View more

Here are a couple of questions: how do you choose to operate as a data scientist? And more importantly, why does this matter? Let’s start with the second question. Over and over, we hear that organizations are struggling to get value from analytics. They know that they need the insights from data to support their decision-making. However, the reality is that they have not yet reached a point where this is happening. Some estimates suggest that over 85% of analytics projects never make it into production. That is, a problem may be identified, and a model developed to address it—but it never becomes part of ‘how we work round here’. Work by Gartner found that data science is often still considered ‘alchemy’ by around 80% of business users. In other words, it is neither understood nor trusted by most people who need to harness its insights.   Developing the story   However, this is only part of the story. In 2021, a paper was presented at the IEEE International Conference on Big Data, about the success factors in data science projects. The authors had surveyed data science professionals about the methodologies they used in implementing data science projects. They found that only 25% followed a formal project implementation methodology.   This chimes with my own experience. There are only a few data science project methodologies out there, and they are not widely used. You could argue that the analytical lifecycle used by SAS promotes a more formal approach to analytical project development. It certainly emphasizes the importance of model deployment as a key part of the project. However, it is not a formal methodology.   This may be the result of how data science is taught at university. You tend to start with personal projects, where you do everything yourself. This is fine when you’re learning—but it really doesn’t work in real life. It follows the hypothesis that you already know the result before you start to work, and means you simply work through from start to finish. However, that doesn’t make any sense in real life, because you don’t know the answer ahead of time. Development tends to be more iterative, and often circular. This top-down approach also leaves very little room to collaborate.   Learning from history?   Those with long memories may find this reminiscent of the situation perhaps 20 or 30 years ago in IT implementation. Missed deadlines, massive budget overruns and general lack of value led to the development of methodologies for project management such as PRINCE. These methodologies are strict, some might even say rigid—but they have proven useful for controlling and managing IT projects effectively over many years. More recently the adoption of Scrum and Agile have given new and more flexible approaches to software development. These methodologies are widely known and used, and generally considered to improve the management of IT and software projects.   These two facts—the struggle to implement data science projects, and the lack of methodology—may be unconnected. After all, not every project needs formal project management methodology to succeed. However, two other aspects were emphasized by the authors of the IEEE paper. First, the most important factors in project success were being able to describe stakeholders’ needs very precisely, being able to communicate project results to end-users, and teamwork. In other words, communication and interpersonal issues, NOT technical factors.   Second, data scientists who used a formal methodology had a much stronger focus on certain aspects of the project. These included the potential risks and, crucially, the route that was needed to put the model into production. This is, of course, the very point at which many data science projects fall.    Changing the culture of data science   Adopting a formal data science project management methodology may or may not be the way to manage every project. I suspect that many do not need to go this far. However, there certainly seems to be room for a greater discussion of how we work as data scientists. For example, how do you share intermediate steps in the development process? How do you communicate your work within larger groups of data scientists, or to end-users? What processes do you have to review and discuss unexpected results?   I think there is a real opportunity here to consider data science work processes as a community. Let’s not get too rigid—but let’s think about how we work, and let’s improve implementation in analytics.   My three key takeaways:   Only 25% of data science professionals surveyed follow a defined process methodology for managing data science projects, highlighting the need for a more defined process. The three most important factors for successful data science projects are project development, teamwork, and data management. Precisely describing stakeholder needs, communicating results to end-users, and team collaboration and coordination are the most important success factors for data science projects. ... View more