Enterprise Session Monitor - How to collect PROC/DATA and Dataset access info without sas-proc-mon
What do I need for this to work?
You need to configure SAS Server Logging for the server type(s) you wish to capture this information from.
<!-- START ESM Agent custom events Logger -->
<logger name="Audit.Data.Dataset" additivity="false">
<appender-ref ref="TimeBasedRollingFile"/>
<level value="Trace"/>
</logger>
<!-- END ESM Agent custom events Logger -->
Make sure the appender ref in the snippet above matches the configuration in your server log XML configuration file.
Got it! What should I do next?
Important Note
Keep in mind that each added logger will increase the number of events captured and stored, and that will increase the ESM database size. Because of this, make sure you only add the custom loggers you really need in your environment. For instance:
- If you only want to capture (PROC/DATA) Steps information, only add markers #1 and #2.
- If you want to capture steps timing information, add #3.
- if you want Dataset access information, add #4-#6.
Once you have decided what you need, follow the steps below:
- Navigate to the ESM Admin Settings -> Log File Markers
- Add one or more of of the following markers:
| # | Marker | Log line filter | Tag Text | Tooltip Text | Marker Color |
| 1 | DATA Step |
.*NOTE:\sDATA\sstatement\sused.*
|
D | DATA |
#23B87C
|
| 2 | PROC Step |
.*NOTE:\sPROCEDURE\s(?<proc>[^\s]+).*
|
P |
PROC ${proc}
|
#23B87C
|
| 3 | Step Timing |
.*(?<type>real|cpu)\stime[\s]+(?<duration>.*)
|
T |
${type} ${duration}
|
#C7B660
|
| 4 | Dataset Read | .*Libref=(?<library>[^\s]+)\sEngine=(?<engine>[^\s]+)\sMember=(?<table>[^\s]+)\sMemberType=(?<type>[^\s]+)\sOpenmode=INPUT\sPath=(?<path>[^\s]+).* | DR |
DR ${library}.${table} [${engine}|${path}]
|
#1F6EB8
|
| 5 | Dataset Write | .*Libref=(?<library>[^\s]+)\sEngine=(?<engine>[^\s]+)\sMember=(?<table>[^\s]+)\sMemberType=(?<type>[^\s]+)\sOpenmode=OUTPUT\sPath=(?<path>[^\s]+).* | DW |
DW ${library}.${table} [${engine}|${path}]
|
#1F6EB8
|
| 6 | Dataset Update | .*Libref=(?<library>[^\s]+)\sEngine=(?<engine>[^\s]+)\sMember=(?<table>[^\s]+)\sMemberType=(?<type>[^\s]+)\sOpenmode=UPDATE\sPath=(?<path>[^\s]+).* | DU |
DU ${library}.${table} [${engine}|${path}]
|
#1F6EB8
|
- Make sure you leave "Filename pattern" and "Hostname pattern" fields blank.
That was easy! How do I test it?
Start a new SAS session and run some code. For instance:
data cars; set sashelp.cars; run;
proc means data=cars; var msrp; run;
proc freq data=cars; run;
That was great! What about Batch Server and batch invocations of the SAS binary?
The configuration above will scan all the log files ESM is aware of for the configured regular expressions and generate events from them. This means if the Agent is properly configured, it will detect the log file for Batch sessions and generate events for them as well. The only thing to keep in mind with the default configuration is that since by default Batch Server writes the standard SAS log output to the logfiles, Dataset Access logger is not enabled, so there will be no Data Access events for the Agent to capture. In other words, if you stop here you will only get DATA/PROC steps information for Batch sessions.
But I want to capture Data Access events for Batch sessions as well
Great! Here is what you will need to do:
- Enable Batch Server logging using a logconfig option
- Enable the Audit.Data.Dataset logger with the Trace level - To do this, add the following logger to your Batch server log configuration XML file, just before the root logger:
<!-- START ESM Agent custom events Logger -->
<logger name="Audit.Data.Dataset" additivity="false">
<appender-ref ref="BatchFileAppender"/>
<level value="Trace"/>
</logger>
<!-- END ESM Agent custom events Logger -->
- Make sure the appender ref in the snippet above matches the configuration in your server log XML configuration file.
- Update ESMLOGCONFIG=1 in the esm-bootstrap.sh script
Great! So I'll also get these events when I run the esm-maintenance utility, right?
Sure thing. They will be in the esm_events_logevents.csv file. See examples from the code snippet used below:
DATA STEP
session_id,time_stamp,pod_id,container_id,hostname,esm_user,event_type,event_level,message,line,log_file_name
1124733630023348001,2023-07-25 18:36:09.159,,,gtp-sas9-vm,hedanc,logEvent,other,DATA,837,
1124733630023348001,2023-07-25 18:36:09.159,,,gtp-sas9-vm,hedanc,logEvent,other,DR WORK.CARS,857,
1124733630023348001,2023-07-25 18:36:09.159,,,gtp-sas9-vm,hedanc,logEvent,other,DW WORK.CARS,834,
1124733630023348001,2023-07-25 18:36:09.158,,,gtp-sas9-vm,hedanc,logEvent,other,DR SASHELP.CARS,831,
PROC MEANS
session_id,time_stamp,pod_id,container_id,hostname,esm_user,event_type,event_level,message,line,log_file_name
1124733630023348001,2023-07-25 18:37:40.543,,,gtp-sas9-vm,hedanc,logEvent,other,PROC MEANS,895,
1124733630023348001,2023-07-25 18:37:40.542,,,gtp-sas9-vm,hedanc,logEvent,other,DR WORK.CARS,891,
PROC FREQ
session_id,time_stamp,pod_id,container_id,hostname,esm_user,event_type,event_level,message,line,log_file_name
1124733630023348001,2023-07-25 18:38:22.513,,,gtp-sas9-vm,hedanc,logEvent,other,DR WORK.CARS,948,
1124733630023348001,2023-07-25 18:38:22.513,,,gtp-sas9-vm,hedanc,logEvent,other,PROC FREQ,952,
References
