Debugging SASCustomer logs - SASCustIntelCore6.5.log

[AravindPalanichamy]

Hi, 

i would like to analyse SASCustIntelCore6.5.log in case of any errors. Lets say i want to grep a column name in the log. I search through the file in unix and can find the particular column which is used in some proc sql step.

Now i want to identify which campaign code this particular column is used? We could not scroll up as this goes through lot of lines. So i lookup above from the place where we find the column name (proc sql step)  for a string "CAMP_CODE=". 

And when we lookup it is not fetching the exact camp_code ran by the user who ran the proc sql step. There are other stored process instance which ran by different user also appear in the middle of the logs. 

Is there a way to debug the SASCustIntelCore logs. Suggestions are welcome. 

Regards,

Aravind

[shill]

There are several techniques for learning things from the log files, but it's pretty hard to connect specific queries to specific campaigns. For starters, you really have to leave the log file set to DEBUG levels, and you need to have the "Tracing Level" (from the Environment Settings tab in CI Studio) set to "High" in order to get the messages you need to attempt this process.

Assuming you have this information, in many cases it's possible to trace the activity from a SQL query log entry by using the session ID (one of the first few columns in the log file) to search upwards in the log for the "dumping task sequence" entries, which will help lead you to which campaign that sequence belongs.

[pcapazzi]

If you are looking for errors reported by SAS I usually search (in a  notepad like app) for ERROR:

What is the database you're connecting to? I know for Teradata there is "query banding".

libname db teradata user=myusr1 password=mypwd1
          QUERY_BAND="org=Marketing;report=Mkt4Q08;";

Teradata could then be queried to find what queries were executed with a given key value pair.

@shill - can the libname QUERY_BAND be set up to pass a parameter that CIS offers such as %campcd and perhaps %mauser.?

[shill]

No, unfortunately not.

I hesitated to point this out because I don't want to set the wrong expectation, but there is a tool that you could ask your local contacts about that may help with analyzing the CI Core log, but it's an "as-is" utility that's not directly supported by SAS...

[pcapazzi]

@shill I've got a follow-up question.

Are the lines in the log one of two types where I'll call them Headers and Detail? Lines that start with a date, time, message type (INFO, DEBUG, ERROR) etc are a header line and the lines without these are considered a detail line. Is it safe to say that all of the detail we see past a header line is relative to the last header only?

I wrote up a bash script to pick up some of the pieces of info from the header lines to cut up the logs into smaller logs. The idea behind this is that one could use grep on the smaller log files for fields or other particular things to see what process was doing it and who was executing it.

split_it.sh:

rm *.log
while read p; do
        if [ "${p:0:5}" = "2018-" ]; then #header lines start with a year
                process=$( echo "$p" | cut -d ' ' -f4 ) #get field 4 after parsing the header line by spaces
                task_id=$( echo "$p" | cut -d ' ' -f5 ) #get field 5
                sasuser=$( echo "$p" | cut -d ' ' -f6 ) #get field 6
                process=${process//\[} #get rid of [
                task_id=${task_id//\[} #get rid of [
                sasuser=${sasuser//\[} #get rid of [
                process=${process//\]} #get rid of ]
                task_id=${task_id//\]} #get rid of ]
                sasuser=${sasuser//\]} #get rid of ]
                redirect=$sasuser$task_id #update the filename the current line will write to
        fi




echo $p >> $redirect.log
done <$1 #operate on the passed to this script as a parameter

I realize the above is not very efficient. It was just to see what the result would be. Also, the SAS User variable would need work because there is a space between the first name and last name.

Thanks.

[AravindPalanichamy]

Thanks pcapazzi. 

This would help me in reduce and filter the logs to analyse further. Thanks for your support

[shill]

That's a mostly-safe categorization. There are some lines where the detail starts on the line below, and some "details" are XML, some are SAS logs, and others are just a dump of information. And most of the lines are stand-alone log entries (what you labelled headers).

There are also strategies you can use by altering the logging xml config file "SASCustIntelCore-log4j.xml" in the Lev1\Web\Common\LogConfig directory by defining new output files for only certain logging contexts, which might allow you to get some smaller files to work with if you're only looking for certain types of messages. Depending on which contexts you include you still may have to contend with the header/details concepts.