Mr.Gwooton,
I had some problems with the quoting 😉
This works for the Update CAS Audit Data, we have a lot of records, I also could have used createdBy,sas.audit.
python3 deletejobhistory.py -e -f 'eq(jobRequest.name,"Update CAS Audit data")' -p 1000 -l 100 -w
Thank you, we will add 72 hours or so, to our jobs that don't have an expiry set.
I will also take a look at the orphan script.
Regards
Dik
Mr. Gwooton
I still have some job related entries for example :
select count(*) from workloadorchestrator.job_std_output ; count --------- 4495370 (1 row)
SharedServices=# select count(*) from workloadorchestrator.job_std_output where line_txt like '%App.ComputeServer.Communication.HTTP%'; count --------- 4266126 (1 row)
SharedServices=# SELECT pg_size_pretty(pg_total_relation_size('workloadorchestrator.job_std_output')) AS table_size; table_size ------------ 3118 MB (1 row)
|
I cannot find a logger with the name App.ComputeServer.Communication.HTTP
sas-viya -p $ns workload-orchestrator logs levels get | grep Loggers |awk -F"name:" '{for (i = 1; i <= NF; i++) {print $i}}'| grep -v NULL
Loggers [map[level:INFO
App.NLS] map[level:ERROR
App.tk.HTTPC.Conn] map[level:FATAL
App.tk.HTTPC.State] map[level:FATAL
App.tk.HTTPC.debug] map[level:FATAL
App.tk.jwtkeystore] map[level:INFO
App.tk.HTTPC.DNSCache] map[level:FATAL
App.tk.eam.rsa.Buffer] map[level:FATAL
App.tk.eam.ssl.openssl111] map[level:ERROR
App.Grid.SGMG.Log.Config.JSON] map[level:FATAL
App.SQLServices.RTE.Statement] map[level:FATAL
App.SQLServices.RTE.Environment] map[level:FATAL
App.SQLServices.RTE.Connection.DCM] map[level:FATAL
App.SQLServices.RTE.Statement.POSTGRES.stats]]
These records are not for jobs from the jobExecution service (those that show in the Monitoring tab of the Jobs and Flows page in SAS Environment Manager) but records stored by SAS Workload Orchestrator related to its job history. This would include anything launched to SWO, so would include all compute server, connect server and batch servers that result in a pod being created. The table you are looking at, job_std_output, captures a row for each line returned to stdout from each job up to the maxStdOutLines configuration in SWO.
Removing these entries would remove the job history from the Workload Orchestrator page in Environment Manager. There is no function within SWO to remove these entries, so if you wanted to remove old entries from this table you would need to manipulate the tables in the database directly rather than through any interface Viya provides.
If you engage SAS Technical Support, they can provide you with a script you can run on a SAS Workload Orchestrator StatefulSet pod to remove old entries.
The script accepts as arguments a retention period in days and limit in the total number of job records to process, and it builds and executes a SQL query to:
Because this can take a long time to finish, the script nohup's the SQL query so you don't need to remain connected to the pod while it runs.
The SAS Users Group for Administrators (SUGA) is open to all SAS administrators and architects who install, update, manage or maintain a SAS deployment.
SAS technical trainer Erin Winters shows you how to explore assets, create new data discovery agents, schedule data discovery agents, and much more.
Find more tutorials on the SAS Users YouTube channel.