Hello Everyone, If you are a SAS Viya platform user, specially working with Model Management and other APIs, it is very likely  you've come across and even used the amazing Python SASCTL package. But, if you were an R developer, or had to push R models to SAS, you would have to switch between R and Python, or build your own API calls. This could be time consuming and cumbersome. Struggle no more. I am excited to announce the R SASCTL package, where you will be able to interact easily with the SAS Viya platform APIs and of course, manage models straight from R.   Now, prepare your `<-` assignments and let's dive in how to use the package.   Install the package   Since the package is not available on the main R repository, CRAN, you will install from our GitHub release. For additional installation methods and other details check the documentation page.   ## Installing dependencies install.packages(c("jsonlite", "httr", "uuid", "furrr", "ROCR", "reshape2")) ## installing the package ## for this first release we will be using X.X.X = 0.6.2 install.packages("https://github.com/sassoftware/r-sasctl/releases/download/X.X.X/r-sasctl_X.X.X.tar.gz", type = "source", repos = NULL) ## loading the library library("sasctl")   Basic usage   As usual, when dealing with APIs, the first thing we are going to do is authenticate to the SAS Viya server. There are many methods available for this. Here, I will use the most basic one, password authentication. For other methods, such as using an "authinfo" file, authentication code or clients refer to the documentation. There is also a SAS Users blog post:  Authentication to SAS Viya: a couple of approaches, which details the matter.   sess <- session(hostname = "https://myserver.sas.com", username = "username", password = "s3cr3t!")   The most important object we have to keep in mind is "sess" , since it is going to give your authentication information to most functions from now on.   A basic capability of the package is the convenient functions to call SAS Viya platform APIs, called vGET, vPUT, vPOST and vDELETE. These function will not only facilitate API authentication, but also parse the json files to tables, for easier access.   As a simple example, I will use the Folders API. First, we refer to the session, then we provide additional information.   folders <- vGET(sess, path = "folders/folders/") print(folder(names)) [1] "version" "accept" "count" "start" [5] "limit" "name" "items" "links"   As you can see, the response is a simple list with many objects inside it. Most of the information on this first level is about the API call. The actual results and information about the folders are inside the items object.   # showing the first 5 folders head(folders$items[,c("id","name",'memberCount','description')]) id name memberCount description 1 00157c78-9b03-4fd3-be93-817809429e92 Code 5 <NA> 2 002c2d3f-e003-4b3d-9e00-34bff9f4a5ea formats 2 <NA> 3 002dae4a-e256-4ecb-b65d-f0b9b02ebd4e GitSettings 1 <NA> 4 00383427-5773-4ef8-9929-f10bc16ca9ed cdisc-cdash 4 <NA> 5 005923b2-07b9-412e-8459-c2b0cedd7832 Snippets 0 <NA> 6 00661c05-c785-4a65-bceb-f3a80649d57e My Snippets 0 My Snippet   To create a new folder, we make a simple post with the following call.   newFolder <- vPOST(sess, path = paste0("folders/folders/"), query = list(parentFolderUri = folders$items$parentFolderUri[1]), payload = list(name = "newFolder"), httr::content_type("application/json")) ## printing the folder and omitting links because they would use a lot of space. newFolder[-11] $creationTimeStamp [1] "2023-02-07T18:20:08.67038Z" $createdBy [1] "username" $modifiedTimeStamp [1] "2023-02-07T18:20:08.67038Z" $modifiedBy [1] "username" $version [1] 1 $id [1] "e359098b-6020-4068-931a-692b74f091c1" $name [1] "newFolder" $parentFolderUri [1] "/folders/folders/5893970f-701a-4529-b2c3-7968aa3ec46a" $type [1] "folder" $memberCount [1] 0 $etag [1] "W/\"1675794008670380000\""    And finally, to delete the folder, we send a delete call.   deletedFolder <- vDELETE(sess, path = paste0("folders/folders/", newFolder$id)) The resource folders/folders/e359098b-6020-4068-931a-692b74f091c1 was successfully deleted.   Using these methods you can interact with any SAS Viya platform API. Now, let's move to a more interesting use case, where we can interact with SAS Model Manager.   SAS Model Manager and R   When working with SAS Model Manager from R, you can register, publish and manage models. But there are some restrictions on what runs directly in SAS and what may require a prior translation. Right now you can register pure R models, astores (SAS models saved from Viya using tools such as R SWAT or other GUI tools), PMML models or SPK (from SAS Enterprise Miner).   Even though you can register all of them, as of today, on SAS Viya 2023.1, you can publish and execute SAS formats (astores and SPK) in all available destination, but R models will only run on CAS or containers, and PMML (version 4.2) will be automatically translated to SAS code. You can reference the complete table here.   R SASCTL is not yet as advanced as it's Python counterpart to automatically create the scoring code in the format that SAS Viya expects, but read on to see how we can do it.   Train the R model   ## Obtaining our data hmeq <- read.csv("https://support.sas.com/documentation/onlinedoc/viya/exampledatasets/hmeq.csv") ## Cleaning our table hmeq[hmeq == ""] <- NA hmeq <- na.omit(hmeq) ### probably you do not want to do that, but for sake of simplicity hmeq$BAD <- as.factor(hmeq$BAD) hmeq$REASON <- as.factor(hmeq$REASON) hmeq$JOB <- as.factor(hmeq$JOB) ### creating train/test/val partition <- sample(c(1,2,3), replace = TRUE, prob = c(0.7, 0.2, 0.1), size = nrow(hmeq)) ### logistic regression model1 <- glm(formula = BAD ~ ., family = binomial(link = 'logit'), data = hmeq[partition == 1,] ) ## stepwise selection model1 <- MASS::stepAIC(model1, trace = 0) ### model summary summary(model1) Call: glm(formula = BAD ~ JOB + DEROG + DELINQ + CLAGE + NINQ + CLNO + DEBTINC, family = binomial(link = "logit"), data = hmeq[partition == 1, ]) Deviance Residuals: Min 1Q Median 3Q Max -1.8321 -0.4002 -0.2723 -0.1815 3.4436 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -5.112504 0.557474 -9.171 < 2e-16 *** JOBOffice -0.347496 0.328065 -1.059 0.28950 JOBOther 0.218764 0.258648 0.846 0.39767 JOBProfExe 0.227723 0.293421 0.776 0.43769 JOBSales 1.289498 0.667405 1.932 0.05335 . JOBSelf 0.740324 0.493300 1.501 0.13342 DEROG 0.756373 0.125728 6.016 1.79e-09 *** DELINQ 0.797044 0.083996 9.489 < 2e-16 *** CLAGE -0.007848 0.001336 -5.872 4.30e-09 *** NINQ 0.127731 0.043009 2.970 0.00298 ** CLNO -0.019821 0.009297 -2.132 0.03301 * DEBTINC 0.100709 0.012481 8.069 7.08e-16 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: 1397.5 on 2330 degrees of freedom Residual deviance: 1053.8 on 2319 degrees of freedom AIC: 1077.8 Number of Fisher Scoring iterations: 6   Note that R automatically transforms the categorical variables in dummies with a reference level. Verify that it matches the input table. Using a dummy/one-hot-encoder methods to have more control may be useful. Or, you could add a step for correctly treating the data before scoring.   Create a folder to save all the required model files   # Creating a folder to save model information dir.create("myModel") path <- "myModel/"   Save the model and create additional files   Save the model as an standard .rda file ## Saving the model saveRDS(model1, paste0(path, 'rlogistic.rda'), version = 2)   Score the model   Now, to create performance metrics the way the SAS Viya platform expects, we have to score our table.    ## scoring the whole table P_BAD1 <- predict(model1, newdata = hmeq, type = 'response') P_BAD0 <- 1 - P_BAD1 # factors starts as 1 when using as.numeric, # we have to add -1 to move to 0,1 scale correctly # since diagnostics expects a numeric value scoreddf <- data.frame(BAD = as.numeric(hmeq$BAD) - 1, P_BAD1 = P_BAD1, P_BAD0 = P_BAD0, partition = partition)   and then we can create all the three basic diagnostic files ROC, LIFT and Fit statistics at once.   diags <- diagnosticsJson(validadedf = scoreddf[scoreddf$partition == 3,], traindf = scoreddf[scoreddf$partition == 1,], testdf = scoreddf[scoreddf$partition == 2,], targetEventValue = 1, targetName = "BAD", path = path) [1] "File written to myModel/dmcas_lift.json" [1] "File written to myModel/dmcas_roc.json" [1] "File written to myModel/dmcas_fitstat.json"   As mentioned earlier, R SASCTL doesn't yet have the feature to automatically create the score code; however, it can create a sample of the structure in the path, so you can edit to match your needs.   create_scoreSample(path) Example file copied to myModel/scoreCode.R   Note: if you are using RStudio, it will automatically open the file.   Next, replace the created file code with the following code, which will allow SAS to run the model on CAS.   scoreFunction <- function(LOAN, MORTDUE, VALUE, REASON, JOB, YOJ, DEROG, DELINQ, CLAGE, NINQ, CLNO, DEBTINC) { #output: P_BAD0, P_BAD1, BAD #rdsPath = './' ## uncomment this for testing in local if (!exists("rlogistic.rda")) { assign("model1", readRDS(file = paste(rdsPath, 'rlogistic.rda', sep = '')), envir = .GlobalEnv) } data <- data.frame(LOAN = LOAN, MORTDUE = MORTDUE, VALUE = VALUE, REASON = REASON, JOB = JOB, YOJ = YOJ, DEROG = DEROG, DELINQ = DELINQ, CLAGE = CLAGE, NINQ = NINQ, CLNO = CLNO, DEBTINC = DEBTINC) ### scorng new data P_BAD1 <- predict.glm(model1, newdata = data, type = 'response') P_BAD1 P_BAD0 <- 1 - P_BAD1 # this is P_BAD0 BAD <- ifelse(P_BAD1 >= 0.4, 1, 0) ### removing names to avoid additional info in the output list names(P_BAD0) <- NULL names(P_BAD1) <- NULL names(BAD) <- NULL # Include scoring logic here to get a list of the output variables. output_list <- list('P_BAD0' = P_BAD0, 'P_BAD1' = P_BAD1, 'BAD' = as.character(BAD)) return(output_list) }   Create model files   Further, we can create the last files to configure model manager when uploading the model.   ## writing other files write_in_out_json(hmeq[,-1], input = TRUE, path = path) write_in_out_json(scoreddf[-4], input = FALSE, path = path) write_fileMetadata_json(scoreCodeName = "scoreCode.R", scoreResource = "rlogistic.rda", path = path) write_ModelProperties_json(modelName = "Rlogistic", modelFunction = "Classification", trainTable = "hmeq", algorithm = "Logistic Regression", numTargetCategories = 2, targetEvent = "1", targetVariable = "BAD", eventProbVar = "P_BAD1", modeler = "sasctl man", path = path)   Register the model   Now, we zip our files and register the model.   files_to_zip <- list.files(path, "*.json|*.R|*.rda", full.names = T) zip(paste0(path, "Rmodel.zip"), files = files_to_zip) mod <- register_model( session = sess, file = "myModel/Rmodel.zip", name = "RLogistic", type = "zip", project = "R_sasctl", force = TRUE ) The project with the name R_sasctl has been successfully created   Verify model in SAS Viya   We can go to SAS Model Manager and check for our model and its statistics.           Since this post is quite long at this point, I'll point you to the register_model function documentation for additional examples and scoring methods. The process is the same, just with different API calls. Be sure to refer back to the documentation and repository often, as we'll offer more examples as they're created.   Finally While functionality and example for R SASCTL will continue to grow over time, the package already offers a wide variety of functions for creating and managing models. Please, feel free to reach me for any comments, questions, or feedback. Thanks a lot for the ride!     ... View more

Bom dia pessoal! Vamos começar a semana depois do feriado com uma novidade quentinha! O SAS Viya está vindo com um belo upgrade, agora é possível usar python direto so SAS Studio. E claro, de brinde ainda pode ser usado dentro do Studio Flow também. Segue o link pra quem quiser saber mais [EN]: https://blogs.sas.com/content/subconsciousmusings/2021/11/08/sas-studio-python-editor/ E claro, qualquer dúvida estou aqui para ajudá-los 🙂 Abraço ... View more

Olá pessoal,   Modelos anlíticos vem em vários formatos, tamanhos e, complexidades. Um uso comum do software do SAS é utilizá-lo para para manejar o ciclo de vidas de modelos. Entretanto, reconhecemos que nem sempre este é o caso. Você pode precisar utilizar as saídas do SAS fora do SAS. Seja criando modelos em Python ou R e colocando em produção no SAS ou criando modelos no SAS e utilizá-los em outro lugar. Para ajudar no último caso, estou aqui para apresentá-los aos mais novos pacotes Open Source para Python, sas-scoring-translator-python (pysct), e para o R, sas-scoring-translator-r (rsct), que traduz os "scoring codes" do SAS para estas linguagens. Isso facilita o uso de modelos SAS na sua aplicação (ou caso você queira aprender um pouco mais como o pacote  SWAT funciona). Estas ferramentas estão disponíveis no sassoftware GitHub.   Obtendo o código de escoragem SAS A essa altura vocês já devem saber que o SAS possui várias interfaces que ajudam a criar modelos (que podem até serem feitos para você através do Auto ML). E, claro, depois de criar seus fabulosos modelos, você quer utilizá-los, o que você poderia fazer então? Exportar o código de escoragem, é claro.   Primeiramente, no Model Studio (SAS Viya), vá até "Pipeline Comparison", como pode assinala a imagem abaixo.     Em seguida, selecione um dos seus modelos, (que não precisa estar registrado ou publicado), e exporte o código de escoragem.       Você irá baixar um arquivo zip com o seu código de escoragem SAS (guarde o nome dmcas_epsscorecode.sas por enquanto, vamos utilizá-lo mais tarde).     Você não precisas descompactar o arquivo (unzip), mas, se você olhar o que tem dentro vai encontrar alguns arquivos .sas parecido com o exemplo abaixo, que vai depender do modelo que você vai utilizar.   /* * This score code file references one or more analytic stores that are located in the caslib "Models". * This score code file references the following analytic-store tables: * _28LWD4IVTS9I294A6F893FNBY_ast */ /*----------------------------------------------------------------------------------*/ /* Product: Visual Data Mining and Machine Learning */ /* Release Version: V2021.1.1 */ /* Component Version: V2020.1.5 */ /* CAS Version: V.04.00M0P05162021 */ /* SAS Version: V.04.00M0P051621 */ /* Site Number: 70180938 */ /* Host: sas-cas-server-default-client */ /* Encoding: utf-8 */ /* Java Encoding: UTF8 */ /* Locale: en_US */ /* Project GUID: f8b37e46-c893-4cf5-8183-0802c729b1b0 */ /* Node GUID: 25d24897-fad4-4e1d-bc94-1a7d93340ade */ /* Node Id: 28LWD4IVTS9I294A6F893FNBY */ /* Algorithm: Gradient Boosting */ /* Generated by: sasdemo */ /* Date: 26JUL2021:19:12:08 */ /*----------------------------------------------------------------------------------*/ data sasep.out; dcl package score _28LWD4IVTS9I294A6F893FNBY(); dcl double "P_BAD1" having label n'Predicted: BAD=1'; dcl double "P_BAD0" having label n'Predicted: BAD=0'; dcl nchar(32) "I_BAD" having label n'Into: BAD'; dcl nchar(4) "_WARN_" having label n'Warnings'; dcl double EM_EVENTPROBABILITY; dcl nchar(12) EM_CLASSIFICATION; dcl double EM_PROBABILITY; varlist allvars [_all_]; method init(); _28LWD4IVTS9I294A6F893FNBY.setvars(allvars); _28LWD4IVTS9I294A6F893FNBY.setkey(n'C08002467175AC235F1C68321869975F6170F229'); end; method post_28LWD4IVTS9I294A6F893FNBY(); dcl double _P_; if "P_BAD0" = . then "P_BAD0" = 0.8005033557; if "P_BAD1" = . then "P_BAD1" = 0.1994966443; if MISSING("I_BAD") then do ; _P_ = 0.0; if "P_BAD1" > _P_ then do ; _P_ = "P_BAD1"; "I_BAD" = ' 1'; end; if "P_BAD0" > _P_ then do ; _P_ = "P_BAD0"; "I_BAD" = ' 0'; end; end; EM_EVENTPROBABILITY = "P_BAD1"; EM_CLASSIFICATION = "I_BAD"; EM_PROBABILITY = MAX("P_BAD1", "P_BAD0"); end; method run(); set SASEP.IN; _28LWD4IVTS9I294A6F893FNBY.scoreRecord(); post_28LWD4IVTS9I294A6F893FNBY(); end; method term(); end; enddata;   Até então tudo certo, desde que você saiba utilizar códigos SAS. Entretanto, se você não souber, ou caso queira escorar tabelas utilizando todo o poder de processamento distribuído do SAS Viya através do R ou Python, um código SAS apenas não te ajuda exatamente. Claro, você poderia usar o SWAT  e fazer toda a tradução manualmente, mas isso não seria nada eficiente. E é para isso que estamos aqui! O pysct - Python Scoring Code Translator (e rsct) vão te ajudar. Ele vai ler o arquivo zip e traduzir para você, vamos dar uma olhada.   Vamos começar com Python, mas caso esteja interessado apenas em R, sinta-se a vontade para pular até a sessão do R.   Python Scoring Code Translator   Primeiramente, vamos instalar o pacote a partir do GitHub do SAS:   ## Instale direto do github caso ainda não o tenha pip install git+https://github.com/sassoftware/sas-scoring-translator-python.git   A ferramenta é bem direta e fácil de utilizar. Olhe a tabela de referência, de onde seu modelo veio e o tipo do cógido de escoragem. No nosso exemplo nos fizemos um modelo através do Model Studio, e, o tipo de código de escoragem é o nome que destaquei mais cedo, dmcas_epscorecode.sas. Com isso, a função que utilizaremos é a EPS_translate(). Outras combinações de modelos e tipo de código de escoragem estão definidos na tabela a seguir:   Interface Code Type Base File Name Translation Function Model Studio DataStep dmcas_scorecode.sas pysct.DS_translate() Model Studio DS2 dmcas_epscorecode.sas pysct.EPS_translate() Visual Text Analytics Sentiment - CAS Procedure scoreCode.sas pysct.nlp_sentiment_translate() Visual Text Analytics Categories - CAS Procedure scoreCode.sas pysct.nlp_category_translate() Visual Text Analytics Topics - CAS Procedure AstoreScoreCode.sas pysct.nlp_topics_translate() Visual Text Analytics Concepts - CAS Procedure ScoreCode.sas pysct.nlp_concepts_translate()   E, com apena a código a seguir, você já terá tudo o que precisa. No nosso exemplo, veja o código:   # importando a biblioteca import pysct out = pysct.EPS_translate( in_file = "C:/score_code_Gradient Boosting.zip", ## caminho do seu .zip (sim, .zip compactado, não precisa se preocupar) out_caslib = "casuser", ## Nome da caslib da tabela de saída (depois dos dados escorados) out_castable = "hmeq", ## Nome da tabela de saida (depois dos dados escorados) in_caslib = "public", ## Nome da caslib onde a tabela a ser escorada está in_castable = "hmeq", ## Nome da tabela a ser escorada copyVars="ALL", ## por padrão, o SAS retorna apenas o resultados, utilize "ALL" caso queira copiar todas as variáveis, ou omita caso contrário out_file="gradientBoosting.py" ## caminho de saída do arquivo python) out.keys()   Exemplo do output: The file was successfully written to gradientBoosting.py dict_keys(['ds2_raw', 'py_code', 'out_caslib', 'out_castable', 'out_file'])   Por padrão, o pysct escreve o arquivo no seu diretório atual. Todo o código pode ser encontrado no objeto  out  no caso de querer mais detalhes. Mas agora vamos dar uma olhada no código gerado, o  gradientBoosting.py .   ## SWAT package needed to run the codes, below the packages in pip and conda # documentation: https://github.com/sassoftware/python-swat/ # pip install swat # conda install -c sas-institute swat import swat ## Defining tables and models variables in_caslib = "public" in_castable = "hmeq" out_caslib = "casuser" out_castable = "hmeq" astore_name = "_28LWD4IVTS9I294A6F893FNBY_ast" astore_file_name = "_28LWD4IVTS9I294A6F893FNBY_ast.sashdat" ## Connecting to SAS Viya conn = swat.CAS(hostname = "myserver.com", ## change if needed port = 8777, protocol='http', ## change protocol to cas and port to 5570 if using binary connection (unix) username='username', ## use your own credentials password='password') ## we encorage using .authinfo ## Loading model to memory ## assuming the model is already inside the viya server conn.table.loadTable(caslib= "Models", path = astore_file_name, #case sensitive casOut = {"name": astore_name, "caslib": "Models"} ) score_table = conn.CASTable(name = in_castable, caslib = in_caslib ) column_names = score_table.columns.tolist() ## loading astore actionset and scoring conn.loadActionSet("astore") conn.astore.score(table = {"caslib": in_caslib, "name": in_castable}, out = {"caslib": out_caslib, "name": out_castable, "replace": True}, copyVars = column_names, rstore = {"name": astore_name, "caslib": "Models"} ) ## Obtaining output/results table scored_table = conn.CASTable(name = out_castable, caslib = out_caslib) scored_table.head()   E a mágica está feita, você precisa apenas editar a conexão ( swat.CAS ) com as suas credencias e do seu servidor, e o seu código está pronto para ser usado no Python. Apesar de utilizar alguns valores padrões (ou copiar do código SAS original), você pode alterar para ajustar ao seu caso, mas agora você já tem uma boa ideia de como começar a criar ótimas integrações.   R Scoring Code Translator Se você leu a sessão do Python, não há muitas mudanças, apenas na sintaxe. Preparem suas <- !   Primeiramente, vamos instalar o pacote do GitHub do SAS (não está disponível no CRAN).   # Como o pacote não está disponível no CRAN, você deve instalá-lo a partir do git # recomendamos utilizar o pacote remotes # install.packages("remotes") # descomente em caso não o tenha instalado remotes::install_github("sassoftware/sas-scoring-translator-r")   Para saber exatamente qual função utilizar, utilize a tabela de referência para comparar de onde tirou seu modelo, e qual é o tipo do código de escorage. No nosso exemplo, o modelo foi feito no Model Studio, e o tipo do código é o que pedi para guardarem mais cedo, dmcas_epscorecode.sas . Assim, basta utilizar a função EPS_translate(). Outras combinações estão descritas na tabela a seguir.   Interface Code Type Base File Name Translation Function Model Studio DataStep dmcas_scorecode.sas DS_translate() Model Studio DS2 dmcas_epscorecode.sas EPS_translate() Visual Text Analytics Sentiment - CAS Procedure scoreCode.sas nlp_sentiment_translate() Visual Text Analytics Categories - CAS Procedure scoreCode.sas nlp_category_translate() Visual Text Analytics Topics - CAS Procedure AstoreScoreCode.sas nlp_topics_translate() Visual Text Analytics Concepts - CAS Procedure ScoreCode.sas nlp_concepts_translate()   E, com algumas linhas, conseguimos traduzir o nosso código, sem nem precisar descompactá-lo.   ## Carregando o pacote library("rsct") output_infos <- EPS_translate(in_file = "C:/score_code_Gradient Boosting.zip", ## caminho do seu .zip (sim, .zip compactado, não precisa se preocupar) out_caslib = "casuser", ## Nome da caslib da tabela de saída (depois dos dados escorados) out_castable = "hmeq_scored", ## Nome da tabela de saida (depois dos dados escorados) in_caslib = "public", ## Nome da caslib onde a tabela a ser escorada está in_castable = "hmeq", ## Nome da tabela que a ser escorarada copyVars = "ALL", ## por padrão, o SAS retorna apenas o resultados, utilize "ALL" caso queira copiar todas as variáveis, ou omita caso contrário out_file = "gb_translated.R" ## caminho de saída do arquivo R ) names(output_infos)   Exemplo do output: File successfully written to gb_translated.R [1] "r_code" "out_file" "out_caslib" "out_castable"   O arquivo  gb_translated.R foi salvo no diretório padrão de sua sessão, mas você poderia também utilizar um caminho completo. O objeto  output_infos é uma lista com detalhes da tradução, caso você precise utilizar em outro lugar. E esse é o código gerado:   ## install swat package from github if needed, uncomment OS version # install.packages('https://github.com/sassoftware/R-swat/releases/download/v1.6.1/R-swat-1.6.1-linux64.tar.gz',repos=NULL, type='file') ## linux # install.packages('https://github.com/sassoftware/R-swat/releases/download/v1.6.1/R-swat-1.6.1-win64.tar.gz',repos=NULL, type='file') ## windows # install.packages('https://github.com/sassoftware/R-swat/releases/download/v1.6.1/R-swat-1.6.1-REST-only-osx64.tar.gz',repos=NULL, type='file') ## osx ## Load library library("swat") ## Defining tables and models variables in_caslib <- "public" in_castable <- "hmeq" out_caslib <- "casuser" out_castable <- "hmeq_scored" astore_name <- "_28LWD4IVTS9I294A6F893FNBY_ast" astore_file_name <- "_28LWD4IVTS9I294A6F893FNBY_ast.sashdat" ## Connecting to SAS Viya conn <- CAS(hostname = "myserver.com", ## change if needed port = 8777, protocol='http', ## change protocol to cas and port to 5570 if using binary connection (unix) username='sasusername', ## use your own credentials password='password') ## we encorage using .authinfo ## Loading model to memory cas.table.loadTable(conn, caslib= "Models", path = astore_file_name , #case sensitive casOut = list(name = astore_name, caslib = "Models") ) ## Defining scoring table obtaining column names score_table <- defCasTable(conn, tablename = in_castable, caslib = in_caslib) column_names <- names(score_table) ## loading astore actionset and scoring loadActionSet(conn, "astore") cas.astore.score(conn, table = list(caslib= in_caslib, name = in_castable), out = list(caslib = out_caslib, name = out_castable, replace = TRUE), copyVars = column_names, rstore = list(name = astore_name, caslib = "Models") ) ## Obtaining output/results table scored_table <- defCasTable(conn, tablename = out_castable, caslib = out_caslib) head(scored_table)   Como você pode notar, está quase pronto para ser utilizado, só falta editar a conexão com o CAS (usando  CAS ) com suas credencias e do seu servidor, e seu código está pronto para ser usado diretamente no R. Você pode fazer as alterações que precisar, mas é uma ótima maneira de começar a entender como SAS e Open Source podem trabalhar juntos.   Conclusão   Quanto antes concordarmos que não é uma questão de Python, R ou SAS, mas sim com SAS, mais rápido podemos criar soluções integradas. Eu realmente espero que essas ferramentas possam ser úteis e interessantes para vocês. E é claro, vamos continuar melhorando estes pacotes e criando mais, especialmente se soubermos que vocês estão utilizando. Qualquer report de bug ou pedidos podem ser feitos no GitHub. Vamos continuar melhorando todas as formas de integração. ... View more

Hello everyone, Analytic models come in many shapes, sizes, and complexities. A common SAS use case is to run through the entire Model Management lifecycle with the SAS software; however, we recognize that this is not always the case. You may need to work with SAS objects outside of SAS. Whether that be using models created in Python or R and deploying them to SAS or creating a model in SAS and needing to deploy it elsewhere.   To assist in the latter use case, I am here to present you to the new Open Source packages for Python, sas-scoring-translator-python (pysct), and for R, sas-scoring-translator-r (rsct), which translate SAS scoring code to those languages. This makes it easy to call SAS models from your application (or if you just want to learn a bit more of how the SWAT package works). The tools are available on sassoftware GitHub.   Get the SAS Scoring Code As you might know by now, SAS has a lot of interfaces where you can build models (it can even build one for you with Auto ML). And, after you create your great models you want to use them, so what could you do? Export a scoring code, of course.   First, in Model Studio go to Pipeline comparison as seen below.     Next, select your model (which doesn't have to be published to MAS), and export its scoring code.       This results in a zip file with your SAS scoring code (save the file name dmcas_epsscorecode.sas in your head; we'll use it later on).     You don't have to unzip the file, but if you take a look inside, you will find something like the example below depending on which model you are using.   /* * This score code file references one or more analytic stores that are located in the caslib "Models". * This score code file references the following analytic-store tables: * _28LWD4IVTS9I294A6F893FNBY_ast */ /*----------------------------------------------------------------------------------*/ /* Product: Visual Data Mining and Machine Learning */ /* Release Version: V2021.1.1 */ /* Component Version: V2020.1.5 */ /* CAS Version: V.04.00M0P05162021 */ /* SAS Version: V.04.00M0P051621 */ /* Site Number: 70180938 */ /* Host: sas-cas-server-default-client */ /* Encoding: utf-8 */ /* Java Encoding: UTF8 */ /* Locale: en_US */ /* Project GUID: f8b37e46-c893-4cf5-8183-0802c729b1b0 */ /* Node GUID: 25d24897-fad4-4e1d-bc94-1a7d93340ade */ /* Node Id: 28LWD4IVTS9I294A6F893FNBY */ /* Algorithm: Gradient Boosting */ /* Generated by: sasdemo */ /* Date: 26JUL2021:19:12:08 */ /*----------------------------------------------------------------------------------*/ data sasep.out; dcl package score _28LWD4IVTS9I294A6F893FNBY(); dcl double "P_BAD1" having label n'Predicted: BAD=1'; dcl double "P_BAD0" having label n'Predicted: BAD=0'; dcl nchar(32) "I_BAD" having label n'Into: BAD'; dcl nchar(4) "_WARN_" having label n'Warnings'; dcl double EM_EVENTPROBABILITY; dcl nchar(12) EM_CLASSIFICATION; dcl double EM_PROBABILITY; varlist allvars [_all_]; method init(); _28LWD4IVTS9I294A6F893FNBY.setvars(allvars); _28LWD4IVTS9I294A6F893FNBY.setkey(n'C08002467175AC235F1C68321869975F6170F229'); end; method post_28LWD4IVTS9I294A6F893FNBY(); dcl double _P_; if "P_BAD0" = . then "P_BAD0" = 0.8005033557; if "P_BAD1" = . then "P_BAD1" = 0.1994966443; if MISSING("I_BAD") then do ; _P_ = 0.0; if "P_BAD1" > _P_ then do ; _P_ = "P_BAD1"; "I_BAD" = ' 1'; end; if "P_BAD0" > _P_ then do ; _P_ = "P_BAD0"; "I_BAD" = ' 0'; end; end; EM_EVENTPROBABILITY = "P_BAD1"; EM_CLASSIFICATION = "I_BAD"; EM_PROBABILITY = MAX("P_BAD1", "P_BAD0"); end; method run(); set SASEP.IN; _28LWD4IVTS9I294A6F893FNBY.scoreRecord(); post_28LWD4IVTS9I294A6F893FNBY(); end; method term(); end; enddata;     This looks fine, as long you know enough SAS. However, if you don't, or if you want to score tables with the distributed power of SAS Viya using Python or R, this doesn't really help you. Of course, you could use SWAT and translate all this by hand, but this would not be efficient or quick. This is why we are here! The pysct - Python Scoring Code Translator (and rsct) will help you. It will read the zip file and translate it for you, let's look at it.   We will start with Python, but if you are interested only in R, feel free to jump down to the R section.   Python Scoring Code Translator First, let's install the package from SAS GitHub:   ## Install directly from git if you don't have it pip install git+https://github.com/sassoftware/sas-scoring-translator-python.git   The tool is quite easy to use. Look at the reference table and check where your model came from and the scoring code type. In our example, we've got a model from Model Studio, and the scoring code type is the name I told you earlier to save, dmcas_epscorecode.sas. Now, we use the EPS_translate() function. Other model and scoring code combos are defined in the following table.   Interface Code Type Base File Name Translation Function Model Studio DataStep dmcas_scorecode.sas pysct.DS_translate() Model Studio DS2 dmcas_epscorecode.sas pysct.EPS_translate() Visual Text Analytics Sentiment - CAS Procedure scoreCode.sas pysct.nlp_sentiment_translate() Visual Text Analytics Categories - CAS Procedure scoreCode.sas pysct.nlp_category_translate() Visual Text Analytics Topics - CAS Procedure AstoreScoreCode.sas pysct.nlp_topics_translate() Visual Text Analytics Concepts - CAS Procedure ScoreCode.sas pysct.nlp_concepts_translate()   With just the following line, you will have everything you need. In my case it would look like the code below.   import pysct out = pysct.EPS_translate( in_file = "C:/score_code_Gradient Boosting.zip", ## path to your file (yes, zipped, you don't have to worry) out_caslib = "casuser", ## the caslib of the output table (after data scored) out_castable = "hmeq", ## the table name of the output table (after data scored) in_caslib = "public", ## the caslib table you want to score in_castable = "hmeq", ## the table name of the table you want to score copyVars="ALL", ## by default SAS only returns the scored output, use "ALL" if you want to copy all table vars, or just omit if you don't want to copy out_file="gradientBoosting.py" ## the output file path ) out.keys()   Sample response: The file was successfully written to gradientBoosting.py dict_keys(['ds2_raw', 'py_code', 'out_caslib', 'out_castable', 'out_file'])   By default, pysct writes the file to your current working directory. All of the code can be found in the out object in case you want to see it, but lets take a look in the output gradientBoosting.py .   ## SWAT package needed to run the codes, below the packages in pip and conda # documentation: https://github.com/sassoftware/python-swat/ # pip install swat # conda install -c sas-institute swat import swat ## Defining tables and models variables in_caslib = "public" in_castable = "hmeq" out_caslib = "casuser" out_castable = "hmeq" astore_name = "_28LWD4IVTS9I294A6F893FNBY_ast" astore_file_name = "_28LWD4IVTS9I294A6F893FNBY_ast.sashdat" ## Connecting to SAS Viya conn = swat.CAS(hostname = "myserver.com", ## change if needed port = 8777, protocol='http', ## change protocol to cas and port to 5570 if using binary connection (unix) username='username', ## use your own credentials password='password') ## we encorage using .authinfo ## Loading model to memory ## assuming the model is already inside the viya server conn.table.loadTable(caslib= "Models", path = astore_file_name, #case sensitive casOut = {"name": astore_name, "caslib": "Models"} ) score_table = conn.CASTable(name = in_castable, caslib = in_caslib ) column_names = score_table.columns.tolist() ## loading astore actionset and scoring conn.loadActionSet("astore") conn.astore.score(table = {"caslib": in_caslib, "name": in_castable}, out = {"caslib": out_caslib, "name": out_castable, "replace": True}, copyVars = column_names, rstore = {"name": astore_name, "caslib": "Models"} ) ## Obtaining output/results table scored_table = conn.CASTable(name = out_castable, caslib = out_caslib) scored_table.head()   And the magic is done, you just have to edit the connection ( swat.CAS ) with your credentials and server name, and your code is ready to use in Python. Even though it uses some default values (or copy from your scoring code file), you are free to change things as you fit. At this point though, you have a good starting point for better integration.   R Scoring Code Translator If you read the Python section, there is not much to change, only in syntax. Get your <- keys ready.   First, we will install the package from SAS GitHub (it is not available on CRAN).   # Since the package is not available on cran, you have to install from our git # we recommend using the remotes package # install.packages("remotes") # uncomment if you don't have it yet remotes::install_github("sassoftware/sas-scoring-translator-r")   To know exactly which function to use, take look at the reference table and check where you got your model from, and the scoring code type. In our example, we've got a model from Model Studio, and the scoring code type is the name I told your earlier to remember, dmcas_epscorecode.sas. So, we should just use the EPS_translate() function. Other model and scoring code combos are defined in the following table.   Interface Code Type Base File Name Translation Function Model Studio DataStep dmcas_scorecode.sas DS_translate() Model Studio DS2 dmcas_epscorecode.sas EPS_translate() Visual Text Analytics Sentiment - CAS Procedure scoreCode.sas nlp_sentiment_translate() Visual Text Analytics Categories - CAS Procedure scoreCode.sas nlp_category_translate() Visual Text Analytics Topics - CAS Procedure AstoreScoreCode.sas nlp_topics_translate() Visual Text Analytics Concepts - CAS Procedure ScoreCode.sas nlp_concepts_translate()   And with a couple of lines we will be able to translate our code. We don't even need to unzip our Scoring code.   ## load the package library("rsct") output_infos <- EPS_translate(in_file = "C:/score_code_Gradient Boosting.zip", ## path to your file (yes, zipped, you don't have to worry) out_caslib = "casuser", ## the caslib of the output table (after data scored) out_castable = "hmeq_scored", ## the table name of the output table (after data scored) in_caslib = "public", ## the caslib table you want to score in_castable = "hmeq", ## the table name of the table you want to score copyVars = "ALL", ## by default SAS only returns the scored output, use "ALL" if you want to copy all table vars, or just omit if you don't want to copy out_file = "gb_translated.R" ## the output file path ) names(output_infos)   Sample response: File successfully written to gb_translated.R [1] "r_code" "out_file" "out_caslib" "out_castable"   The  gb_translated.R was written to your working directory, but you could also set a full path. The output_infos is a list with details if you need to use the results somewhere else. Look at the output code:   ## install swat package from github if needed, uncomment OS version # install.packages('https://github.com/sassoftware/R-swat/releases/download/v1.6.1/R-swat-1.6.1-linux64.tar.gz',repos=NULL, type='file') ## linux # install.packages('https://github.com/sassoftware/R-swat/releases/download/v1.6.1/R-swat-1.6.1-win64.tar.gz',repos=NULL, type='file') ## windows # install.packages('https://github.com/sassoftware/R-swat/releases/download/v1.6.1/R-swat-1.6.1-REST-only-osx64.tar.gz',repos=NULL, type='file') ## osx ## Load library library("swat") ## Defining tables and models variables in_caslib <- "public" in_castable <- "hmeq" out_caslib <- "casuser" out_castable <- "hmeq_scored" astore_name <- "_28LWD4IVTS9I294A6F893FNBY_ast" astore_file_name <- "_28LWD4IVTS9I294A6F893FNBY_ast.sashdat" ## Connecting to SAS Viya conn <- CAS(hostname = "myserver.com", ## change if needed port = 8777, protocol='http', ## change protocol to cas and port to 5570 if using binary connection (unix) username='sasusername', ## use your own credentials password='password') ## we encorage using .authinfo ## Loading model to memory cas.table.loadTable(conn, caslib= "Models", path = astore_file_name , #case sensitive casOut = list(name = astore_name, caslib = "Models") ) ## Defining scoring table obtaining column names score_table <- defCasTable(conn, tablename = in_castable, caslib = in_caslib) column_names <- names(score_table) ## loading astore actionset and scoring loadActionSet(conn, "astore") cas.astore.score(conn, table = list(caslib= in_caslib, name = in_castable), out = list(caslib = out_caslib, name = out_castable, replace = TRUE), copyVars = column_names, rstore = list(name = astore_name, caslib = "Models") ) ## Obtaining output/results table scored_table <- defCasTable(conn, tablename = out_castable, caslib = out_caslib) head(scored_table)   As you can see, it is almost ready to use. You just have to edit the CAS connection ( CAS ) with your credentials and server name, and your code is ready to be used in your R environment. You are free to change as you please. This is a good way to start understanding how SAS and Open Source can work together.   Conclusion The sooner we agree on it's not a question of Python, R or SAS, but rather with SAS, the sooner we can move on to creating integrated solutions. I hope that these tools can be useful and interesting for you. We will keep improving this package, specially if I know you are using it. Any bugs or requests, please feel free to open them on Github. Let's keep moving all integration capabilities forward. ... View more