Removal or archiving of older metadata server logs

epniv — Wed, 23 Sep 2020 21:53:12 GMT

We have been accumulating metadata logs and now need to remove or archive those that are not needed. We are planning to use Python to code this out, but I thought I would ask if anyone has come up with a solution. We plan to deploy as a cron job for weekly execution. It's not just a matter of date based file comparison, some PID's can last weeks, others just days.

Using SAS v9.4M5

==================================================

New Metadata logs (see below) are generated on a daily basis, with the date embedded in the filename. Also embedded in the filename is the “process ID” or PID for the latest reboot of the Metadata server. The latest PID can be found in a file within the following directories:

/sasconfig/Lev1/SASMeta/MetadataServer/server.rccsasprd01.pid

/sasconfig/Lev1/ObjectSpawner/server.rccsasprd01.pid

These each contain DIFFERENT PID’s.

Below we can see that both sets of logs had one PID (Object = 48866, Meta = 48731) which then changes after the server was restarted to (Object = 1900, Meta = 1724)

sasinst sasadmin 629048 Sep 14 00:00 ObjectSpawner_2020-09-13_rccsasprd01_48866.log

sasinst sasadmin 455682 Sep 14 16:31 ObjectSpawner_2020-09-14_rccsasprd01_48866.log

sasinst sasadmin 203633 Sep 15 00:00 ObjectSpawner_2020-09-14_rccsasprd01_1900.log

sasinst sasadmin 648740 Sep 16 00:00 ObjectSpawner_2020-09-15_rccsasprd01_1900.log

sasinst sasadmin 22509544 Sep 14 00:00 SASMeta_MetadataServer_2020-09-13_rccsasprd01_48731.log

sasinst sasadmin 16065378 Sep 14 16:31 SASMeta_MetadataServer_2020-09-14_rccsasprd01_48731.log

sasinst sasadmin 6416455 Sep 15 00:00 SASMeta_MetadataServer_2020-09-14_rccsasprd01_1724.log

sasinst sasadmin 22471372 Sep 16 00:00 SASMeta_MetadataServer_2020-09-15_rccsasprd01_1724.log

The general advice from SAS was to keep the two latest sets of PID logs from each server-type. Logs older than that would not be meaningful for any type of error condition, for which SAS would need to review sed logs.

==================================================

Thanks

Re: Removal or archiving of older metadata server logs

Kurt_Bremser — Wed, 23 Sep 2020 22:14:11 GMT

I have a job (shell script) that runs regularly and concatenates object spawner and metadata server logs per day; that takes care of restarts (rare, but they happen).

The concatenated logs are read into SAS datasets and then archived, as I want (and need) to have full documentation who used the service, and when. This is part of GDPR compliance.

topic Re: Removal or archiving of older metadata server logs in Administration and Deployment

Removal or archiving of older metadata server logs

Re: Removal or archiving of older metadata server logs