Using Azure Service Principal to access DataBricks from SAS Viya Application
Recent Library Articles
Recently in the SAS Community Library: The SAS Viya platform supports the Service Principal authentication-based access to the Azure DataBricks database with the CDATA JDBC drivers. This access is available for both SPARK LIBNAME and SPARK CASLIB statements against the Azure Databricks database. SAS' @UttamKumar shows you how to use the Service Principal to include Azure environment details, Databricks properties and more.
on Viya 4 kubernetes, the backup is successful but the purge fails. By getting the log: kubectl logs BackupPurgeJobPod -n Namespace. Error: "Can not perform the purge action for the sas-backup-purge-job configurations,postgres data source as the acknowledgement not received from the backup agent." it happened for a few days in a row. Wondering if happened to anyone, and if so how to fix. Thank you a lot,
... View more
Hi, everyone. I have a dataset with 3 columns, as shown above. The first column, COD_SEG, represents the internal code for each person. The second column, DT_OCORR, represents the date of a specific event (format yymmd7.). The last column,, CIA_ESCOLHIDA, provides a reference for the corresponding COD_SEG and DT_OCORR. My date variable (DT_OCORR) is skipping (jumping) months. I need to insert the missing dates for each COD_SEG. When doing this, I need to fill the other 2 columns with the information from the previous row. Like this. I have: COD_SEG DT_OCORR CIA_ESCOLHIDA 37192 2021-08 1806 37192 2022-02 1806 37192 2022-03 1806 37192 2022-04 1806 37192 2022-06 1806 37192 2022-07 1806 37192 2022-08 1806 37192 2022-09 1806 37192 2022-10 2110 37192 2022-12 2110 37192 2023-01 2110 I need (I have bolded the lines that need to be inserted): COD_SEG DT_OCORR CIA_ESCOLHIDA 37192 2021-08 1806 37192 2021-09 1806 37192 2021-10 1806 37192 2021-11 1806 37192 2021-12 1806 37192 2022-01 1806 37192 2022-02 1806 37192 2022-03 1806 37192 2022-04 1806 37192 2022-05 1806 37192 2022-06 1806 37192 2022-07 1806 37192 2022-08 1806 37192 2022-09 1806 37192 2022-10 2110 37192 2022-11 2110 37192 2022-12 2110 37192 2023-01 2110 My database have so many rows. I need an way to do this for a lot of records. Anyone can help me? I'm using SAS Enterprise Guide.
... View more
I am using SAS/IML to calculate the covariance matrix of data by hand. A problem I encountered was on how to name a bunch of matrices with a common prefix in IML. I summarized my concerns as the two following questions:
(1) Is the horizontal concatenation operator (i.e., "||"), which is valid in the DATA step and the operation of matrix elements of blocks, valid in the naming process of matrices in IML? I was using a macro and was hoping to give my matrices names like cov1, cov2, etc. I tried "cov||1" but the IML reported an error in the log.
(2) How to designate the bunch of matrices with a common prefix in the loading and summation steps? I was using the LOAD statement in IML and attempted to designate the matirces I wished to load with the argument "cov1-cov35". This works in the DATA step, but unfortunately it also did not work in the LOAD statement in IML. So I used the
load _all_;
statement to get round typing the names of every matrix after the "load" clause. But I do not think it a good way if the storage designated is stockpiled with more matrices.
Similarly, I also got into trouble as I was trying to sum all the matrices with the same prefix. This time, there did not seem to be a way to get-around the monotonous typing of matrix names, so I had to type them one by one and join them with the plus sign. But is there a more efficient way?
Thanks!
... View more
Let's suppose I have fitted a model using PROC PHREG, and I now want to compute the survival probabilities. Why do I need to compute them if they are output by SAS? Because we will need to compute the fitted model predictions of survival probabilities on new data, and the way our computer systems are set up, I can't just simply turn over the SAS and say use PROC PLM, the calculation of survival probabilities will be done outside of SAS, so I need to have this other computer system replicate the exact formula.
I get the feeling I'm missing something simple, but I can't figure it out.
So, I make up some stupid data, run PROC PHREG and get a stupid model on the stupid data, obtain the survival probs computed by PROC PHREG, and then can someone guide me through the calculations?
Here's the made up data:
data class;
set sashelp.class;
call streaminit(02785);
bad_rand=rand('uniform');
if bad_rand>0.6 then bad=1;
else bad=0;
if bad=1 then time_to_bad=rand('integer',5,10);
else time_to_bad=10;
run;
Here's PROC PHREG using just one x-variable to make it simple.
proc phreg data=class;
ods output parameterestimates=parms;
model time_to_bad*bad(0)=height;
baseline covariates=class out=baseline cumhaz=cumhaz logsurv=logsurv timelist=5 to 10 by 1 xbeta=xbeta;
output out=survival survival=survival ;
run;
And now here's the output in data set SURVIVAL.
Here is the model output
Here is additional info in the data set BASELINE, I don't know if this is useful to us here.
How is the survival probability for Alfred of 0.914562597 at time_to_bad=6 computed from the model outputs?
... View more
Hello,
I have tried to implement DBSCAN and KNN clustering methods in SAS 9.4. After checking online resources, I found multiple articles on these specific clustering algorithms for SAS Viya but not much for SAS 9.4. Is there anything you could suggest? Currently, I am thinking of using PROC IML to write the algorithm myself. If there is anything ready to use, I would like to try that first.
Thank you!
... View more