@shill I've got a follow-up question.
Are the lines in the log one of two types where I'll call them Headers and Detail? Lines that start with a date, time, message type (INFO, DEBUG, ERROR) etc are a header line and the lines without these are considered a detail line. Is it safe to say that all of the detail we see past a header line is relative to the last header only?
I wrote up a bash script to pick up some of the pieces of info from the header lines to cut up the logs into smaller logs. The idea behind this is that one could use grep on the smaller log files for fields or other particular things to see what process was doing it and who was executing it.
split_it.sh:
rm *.log
while read p; do
if [ "${p:0:5}" = "2018-" ]; then #header lines start with a year
process=$( echo "$p" | cut -d ' ' -f4 ) #get field 4 after parsing the header line by spaces
task_id=$( echo "$p" | cut -d ' ' -f5 ) #get field 5
sasuser=$( echo "$p" | cut -d ' ' -f6 ) #get field 6
process=${process//\[} #get rid of [
task_id=${task_id//\[} #get rid of [
sasuser=${sasuser//\[} #get rid of [
process=${process//\]} #get rid of ]
task_id=${task_id//\]} #get rid of ]
sasuser=${sasuser//\]} #get rid of ]
redirect=$sasuser$task_id #update the filename the current line will write to
fi
echo $p >> $redirect.log
done <$1 #operate on the passed to this script as a parameter
I realize the above is not very efficient. It was just to see what the result would be. Also, the SAS User variable would need work because there is a space between the first name and last name.
Thanks.
