Keep track of who accessed SAS data sets in a compute server

3 Likes

In courses I get asked whether it is possible to track who has accessed which SAS data sets and when. I usually refer them to this article Auditing data access: who did what and when? by Gerry Nelson. You might remember my article Who’s been using my CAS tables? where it is all about CAS tables.

This post will expand on Gerry's article and explain how to use this in SAS Viya with some additional features such as custom message layout and filtering (no log entries for libref WORK or SASHELP).

In a follow up post, I will go into the details how we can add information on the user making the access.

Topics discussed

Basics on specifying logging configuration
Have a custom message layout
How to filter on the custom message layout

Basics on specifying logging configuration

In SAS Viya you can no longer edit the logconfig.xml file and adapt it to your needs. Instead you will use the Configuration page of the SAS Environment Manager to change the logconfig definition or use the sas-viya configuration configurations command line utility. In this article I will show everything using the SAS Environment Manager. See this article Where to configure the SAS Programming Run-time with broader or narrower scope by David Stern for more details on what can be specified where.

To log access to a SAS library we are using the logger name Audit.Data.Dataset as introduced in Gerry's article. We can change the logconfig used by a compute server in the configuration of the compute service. In the SAS Environment Manager this is the place:

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

So we click on the pencil icon and add this additional logger definition:

<!-- Audit.Data.Dataset.Open logger definition -->
<logger name="Audit.Data.Dataset" additivity="false" immutability="true">
  <level value="Trace"/>
  <appender-ref ref="Console"/>
</logger>

Note: we need the level to be trace to get the proper information.

The compute service will automatically restart after the change, allow for some time for the restart to happen. Existing running compute server pods are not affected by the restart of the compute service pod. Only new compute servers created after the restart will be affected. You can check whether the compute service is restarted using this command (age will reflect how long the the compute service has been running):

kubectl -n <your-namespace> get pods -l app=sas-compute
NAME                         READY STATUS  RESTARTS AGE
sas-compute-6c5c54c76d-ngv6w 1/1   Running 0        19s

We can now test the logging by using SAS Studio and run some code. If you already had an existing SAS Studio running, you will need to reset the session. We then run this SAS program within SAS Studio:

%put NOTE: &=systcpiphostname;
libname sugus "!HOME";
data sugus.myclass;
  set sashelp.class;
run;

The macro variable SYSTCPIPHOSTNAME will contain the name of the pod running this compute server. To display the log entries created, we can use this command (we are only interested in the logger name Audit.Data.Dataset):

kubectl -n <your-namespace> logs <your-podname> | grep -i audit.data.dataset | jq

The result will look like this:

{
  "level": "trace",
  "message": "Libref=SASHELP Engine=V9 Member=CLASS MemberType=DATA Openmode=INPUT Path=/opt/sas/viya/home/SASFoundation/sashelp",
  "properties": {
    "caller": "yhaudlog.c:643",
    "logger": "Audit.Data.Dataset.Open",
    "pod": "sas-compute-server-bfed7358-08c8-4b51-81b2-2155cc3b5dec-135",
    "thread": "00000015"
  },
  "source": "compsrv",
  "timeStamp": "2025-01-23T14:36:44.911000+00:00",
  "version": 1
}
{
  "level": "trace",
  "message": "Libref=SUGUS Engine=V9 Member=MYCLASS MemberType=DATA Openmode=OUTPUT Path=/home/christine",
  "properties": {
    "caller": "yhaudlog.c:643",
    "logger": "Audit.Data.Dataset.Open",
    "pod": "sas-compute-server-bfed7358-08c8-4b51-81b2-2155cc3b5dec-135",
    "thread": "00000015"
  },
  "source": "compsrv",
  "timeStamp": "2025-01-23T14:36:44.913000+00:00",
  "version": 1
}

We can compare the message element with the SAS program and can see that sashelp.class has been used as input:

"message": "Libref=SASHELP Engine=V9 Member=CLASS MemberType=DATA Openmode=INPUT Path=/opt/..."

and sugus.myclass has been used as output:

"message": "Libref=SUGUS Engine=V9 Member=MYCLASS MemberType=DATA Openmode=OUTPUT Path=/home/christine"

What we currently cannot see is the user that made the access and some additional information that is available with this logger. If you have the SAS Viya Monitoring for Kubernetes implemented log entries are collected over time and can be easily explored using OpenSearch Dashboards. SAS Viya Monitoring for Kubernetes will add additional information such as user etc.

Have a custom message layout

When we check the documentation about which conversion characters are available to make up the log message we will find the E Conversion Character it can be used together with the Audit.Data.Dataset logger. Beside the default information we have seen above there is more information available like (not all keys listed here):

Key name	Information
Audit.Dataset.Action	reports the action that is being logged (OPEN, DELETE, or RENAME).
Audit.Dataset.Memtype	reports the type of data set member.
Audit.Dataset.Openmode	reports whether the data set is open in READ, WRITE, or UPDATE mode.
Audit.Dataset.Status	reports the status of opening a data set. Valid values are SUCCESS or FAILED.

A custom pattern layout for the log message can be defined in an FilteringAppender. So we are going to define a new Appender with a specific pattern for the log message. The definition looks like this:

<appender name="ConsoleAudit" class="FilteringAppender">
  <appender-ref ref="Console"/>
  <layout>
    <param name="ConversionPattern" value="action=%E{Audit.Dataset.Action} libref=%E{Audit.Dataset.Libref} engine=%E{Audit.Dataset.Engine} member=%E{Audit.Dataset.Member} memtype=%E{Audit.Dataset.Memtype} newmember=%E{Audit.Dataset.NewMember} openmode=%E{Audit.Dataset.Openmode} path=%E{Audit.Dataset.Path} status=%E{Audit.Dataset.Status} sysmsg=%E{Audit.Dataset.Sysmsg} sysrc=%E{Audit.Dataset.Sysrc}"/>
  </layout>
  <param name="PropagateLayout" value="true"/>
</appender>

The PropagateLayout=true ensures, that the new pattern is sent to the appender as %m, the message we see in the log.

We then change the logger definition to use the new appender:

<!-- Audit.Data.Dataset.Open logger definition -->
<logger name="Audit.Data.Dataset" additivity="false" immutability="true">
  <level value="Trace"/>
  <appender-ref ref="ConsoleAudit"/>
</logger>

As before give the compute service some time to restart, reset the SAS Studio session and run a SAS program that will read or write some SAS data set. The log message now looks like this:

"message": "action=OPEN libref=SASHELP engine=V9 member=SUGUS memtype=DATA newmember= openmode=INPUT path=/opt/sas/viya/home/SASFoundation/sashelp status=FAILED sysmsg=ERROR: File SASHELP.SUGUS.DATA does not exist. sysrc=70009"

We can see the additional information in the message. The example provided shows the entry when an input SAS data set is not found.

How to filter on the custom message layout

We now get for every read or write of a SAS data set a log entry. You might want to filter out some log entries based on the name of the libref or some other criteria. To do this, we will use another FilteringAppender. Using this technique, we can ensure, that we only filter log entries coming from the specific logger (Audit.Data.Dataset) and not any generic log message. This new FilteringAppender looks like this:

<appender name="ConsoleAuditFilter" class="FilteringAppender">
  <appender-ref ref="Console"/>
  <filter class="StringMatchFilter">
    <param name="StringToMatch" value="libref=WORK"/>
    <param name="AcceptOnMatch" value="false"/>
  </filter>
  <filter class="StringMatchFilter">
    <param name="StringToMatch" value="libref=SASHELP"/>
    <param name="AcceptOnMatch" value="false"/>
  </filter>
</appender>

This FilteringAppender will work on the pattern layout we defined in the previous FilteringAppender. Here is a picture that shows the flow of a log message.

You can see the final logconfig.xml with all the changes applied.

Summary

We can use the Audit.Data.Dataset logger to log all access to SAS data sets in a SAS library. Using a FilteringAppender we can add additional information to the log message not available in the default log message, as well as filter out log message for specific SAS libraries.

It is recommended to have a log collection utility in place, see Why you need a log and metric monitoring solution for the SAS Viya platform by David Stern for more details. Because once your compute server pod used by SAS Studio is finished you will no longer have access to the log of this pod.

If you are using the SAS Viya Monitoring for Kubernetes. I suggest that you read Export log messages from a command line with getlogs.py and SAS Viya Monitoring for Kubernetes by David Stern on how you can export log messages.

Find more articles from SAS Global Enablement and Learning here.

SAS Communities Library