Hello friends -
Can any one please help with the log cleanup script
I need your help with the Auto log cleanup script for the maintenance of our sas servers. am totally new to unix scripting. can anyone please assist with the cleanup script.
We need to keep the log files 2 months in the /logs directory and any files older than 2 months has to go to the Backup directory and in the Backup directory files should reside for 6 months and any log files older than 6 months should be deleted.
I want this script should perform this cleanup activity in all servers in different /log file locations of the server.
Would really appreciate if you can help with the script
Thank you all in advance
You could probably do this using a couple find commands, something like:
find <log-path> -type f -mtime +60 -exec mv {} <backup-path> \; find <backup-path> -type f -mtime +180 -delete
You could put that in a script and schedule it via cron to run regularly. If you run the command without the actions (i.e. no -delete and no -exec mv {} <backup-path \;) it will return the files it would take the action on.
Thank you gwootton for prompt response..
Actually I found below find commands...but
find /sas/install/config/Lev1/Logs -maxdepth 1 -mtime +60 -type f -exec mv "{}" /backup \;
find /backup -maxdepth 1 -mtime +180 -type f -exec rm "{}" \;
find /sas/install/config/Lev1/Web/WebServer/logs -maxdepth 1 -mtime +60 -type f -exec mv "{}" /sashome/M6_install/config/Lev1/Web/WebServer/logs/backup_062821 \;
find /sashome/M6_install/config/Lev1/Web/WebServer/logs/backup_062821 -maxdepth 1 -mtime +180 -type f -exec rm "{}" \;
find /sas/install/config/Lev1/Web/Logs/SASServer1_1 -maxdepth 1 -mtime +60 -type f -exec mv "{}" /sashome/M6_install/config/Lev1/Web/Logs/SASServer1_1/backup_062821 \;
find /sashome/M6_install/config/Lev1/Web/Logs/SASServer1_1/backup_062821 -maxdepth 1 -mtime +180 -type f -exec rm "{}" \;
but am looking for the script with adding some email messages for each action taking place. find in conjunction with mv or rm. But for every find there must be a communication to sas admin telling what has been done
Script servse the purpose in this case but what is missing the communication and not dealing with the error condition like a situation when there's not enough disk space on the target file system. And also not handling overwriting the file names when moving the file from sources to destinations
I need below three things :
1)need to add comments line for each find command action
2)need to add send email for each action
3) need to add one more condition--- compression of the file after a month
4) also u need to know the size of the file moved from source to destination
and also I need to have a new file name when moving from source to destination
The new name is necessary to avoid overwriting. The new naming convention should reflect the old name + server name + date of the activity,This is necessary since every things goes to same file system And names has to be unique to avoid overwriting
Basically it is a rotation of the log files to compress after a month , followed by moving and then removing
If u do not give new name, it will overwriting in the old file,It us good practice to have any new file with name having time stamp.
That way when we need those files in future we can track them easily
So new file name=old name.time stamp
The whole idea of this practice is that I will be retaining these log file for a year is that i may need them
My opinion is each month. In each folder, compress, tar the previous month's logs
Example; xxxlog-202106.tar.zip at the end of this month
Compression stays in side the source side
After the compression the directory will create a one single file which will be moving after three months
One single file will be Xxxxlog-20320631.targz
And this file will be moved to /log at the end of December
And should have the same name as it is
But should go to the directory for the relevant server inside /log
/log/metadata/xxxlog202106.targz
Would really appreciate your help
Yeah I have some idea about my requirement what the shell script should do, but unfortunately i don't have any idea at all on shell scripting. The find commands also i which sent earlier i found it from google, if anyone can help with my requirement that would be really a big help.
Learning how to write shell scripts isn't really a SAS problem so you will probably find better help on websites devoted to this. Surely your company will have IT people skilled at this and who can help you to come up to speed or who you can ask to write the script for you. I know mine does and I'd be asking them to do the shell scripting for me as like you I know very little about it.
1. I see that you want a rotation of log. Have a look at the UNIX logrotate utility available is most Unix/Linux distributions.
2.The Unix/Linux administrators should be able to help.
3. I fully understand your not knowing shell scripting. Better to learn but not worry. The expectations from SAS Administrators are a growing unrealistically high these days depending upon the environment without much of a career progression.
As soon as you start doing administrator work, Maxim 15 (Know Your Playing Field) turns from a strong recommendation into a requirement.
Knowing at least the basics of bash scripting (creation and use of environment variables, conditional execution, regular expressions, writing to and reading from files, piping, just to name a few) is a must.
You also need to get a grasp on the important UNIX utilities like find, grep, ls, ps. Maxim 6 (Google Is Your Friend) is a big helper in this respect.
Since I do not have to do shell scripting often, I usually need to consult the online documentation for all this myself, so do not be afraid to dig into it.
Hello - Did you have luck in completing the script for SAS log purging? I am also looking at options to set up an automatic SAS LOG purging job to clean up the logs periodically on the windows server. Thank you!
On Windows, if all you want to do is remove orphan SAS WORK libraries then just schedule the cleanup tool in Windows Task Scheduler:
The SAS Users Group for Administrators (SUGA) is open to all SAS administrators and architects who install, update, manage or maintain a SAS deployment.
SAS technical trainer Erin Winters shows you how to explore assets, create new data discovery agents, schedule data discovery agents, and much more.
Find more tutorials on the SAS Users YouTube channel.