Log Analysis

[Doug____]

Is there any way that logs can be analyzed en masse within enterprise guide? I am referring to the individual logs programs/nodes create when executed not the project log.

[RW9]

Not sure exactly what you mean, but you could redirect the log with proc printto log=...; or have proc scaproc run before and at the end of the process (its the source code analyser).  

[andreas_lds]

What exactly does "analyze" mean in this context? Do you want to find warnings/errors or are trying to analyze what happened?

If you want to analyze the logs using another egp, you could extract the egp and and process the log-files without modifying the original code.

[Doug____]

What analyze the logs mean is to have SAS go through all the logs in the project not just for those associated with individual programs and check for the following in addition to warnings and errors to document there are no issues:

Overwritten

Uninitialized

merge by repeats

mathematical operations could not be performed

SQL queries with remerging

unintended numeric to character conversions and vice versa

missing value messages

I do not want to have to manually export anything and PROC PRINTTO only works on the current program. I want the current logs for everything in the project analyzed and parsed and placed into a PDF I can review. I do not want to see any of the notes listed above in the project if at all possible and I do not want to have to manually export anything to do it.

[Patrick]

@Doug____ 

Have you ever noticed that you can look at logs in EG even if the EG client isn't connected to any server (now server active). 

What that must mean is that the logs must reside on the SAS EG client side and not on the SAS server side (where SAS could have access to it). 

If you save an EG project on local disk you get a file with extension .egp, i.e. sample.egp.

Did you know that this is actually a zip archive which you can open with WinZip or the like. If you do so for a bigger project you'll get quite a collection of files and folders. There is on top level a file project.xml which contains the full project organisation and tells you where you can find all the logs.

                        <log>Log-Lq7MPtnVFvkl78Hh</log>

You could also just traverse through the whole archive and pick all the files with extension .log

So now what options do you have to achieve what you want:

Option 1

Route the logs to a SAS server side destination (proc printto).

Comment: Doesn't feel like a very feasible option for you as EG wouldn't get the logs anymore into the project but you would have to access them directly from the server.

Option 2

Use client side software to retrieve the logs from the locally stored .egp file and parse the .log files (you could use PowerShell).

Option 3

Move a copy of your .egp project to the SAS server side and then use SAS to extract the .log files and parse the logs.

Option 4 (not sure)

Save your .egp files in SAS metadata (that's when you choose a server for saving). Use SAS code to query SAS metadata and retrieve the .egp file for processing.

Comment: I believe the .egp files get stored in the SAS content server and I would assume there is some information in SAS metadata with the information where. I have no idea though if this is a feasible option both from a security and accessibility perspective. I also don't know what would happen if you'd run such code out of a project trying to analyse the project itself (locking issues and the like).

I hope this answers your question. Feels all rather involved though.

[andreas_lds]

---

@Doug____ wrote:

What analyze the logs mean is to have SAS go through all the logs in the project not just for those associated with individual programs and check for the following in addition to warnings and errors to document there are no issues:

Overwritten

Uninitialized *

merge by repeats

mathematical operations could not be performed *

SQL queries with remerging

unintended numeric to character conversions and vice versa *

missing value messages *

I do not want to have to manually export anything and PROC PRINTTO only works on the current program. I want the current logs for everything in the project analyzed and parsed and placed into a PDF I can review. I do not want to see any of the notes listed above in the project if at all possible and I do not want to have to manually export anything to do it.

---

Solving this will be a though job. @Patrick  provided a list of ways to go, depending on the place where the projects are stored i would use second or third suggestion or a combination of both: call a script from the sas session extracting the logs and then using sas to parse the text files.

You could use the option dsoptions=note2err turning all unexpected notes (marked with * in your list) into errors, together with errors=1 code execution is prevented unless those signs of laziness are fixed.