topic Re: Keeping the Log Window Clear in ODS and Base Reporting

Keeping the Log Window Clear

deleted_user — Tue, 14 Aug 2007 09:32:06 GMT

Hi All,

I have an odd query that I hope you can help with. I have a fairly large file that I'm processing, my code is fairly straightforward and runs quickly but there are lots of iterations. For example, I'm currently running a test file with 24,130 iterations and I will later have to execute the code on a file with c.195,000.

My problem is that after around 900 iterations the Log window becomes full and needs to be manually cleared. The program is calling a macro so there's no lines of code in the Log window, just notes but there's so many lines of notes that it's filling up.

Is there any way to prevent the notes appearing or to automatically clear the Log window so that the program can be left to run by itself?

Many thanks in advance.

Re: Keeping the Log Window Clear

deleted_user — Tue, 14 Aug 2007 10:40:34 GMT

You can try using Proc Printto to output the log window contents to an external file which can be deleted later .

Re: Keeping the Log Window Clear

Tim_SAS — Tue, 14 Aug 2007 12:25:47 GMT

Why not run your program in batch mode? The log will be written to a file and the only limitation on its size will be how much disk space you have. This way, if something goes wrong, you'll have the log to refer to. If nothing goes wrong, delete the log.

Re: Keeping the Log Window Clear

deleted_user — Tue, 14 Aug 2007 13:36:02 GMT

I hadn't thought of that, although to be fair, I've never actually run anything in Batch mode before.

I'm using v.9.1.3 sp3 for Windows, how do I go about running my program in batch mode?

Re: Keeping the Log Window Clear

Tim_SAS — Tue, 14 Aug 2007 14:17:55 GMT

The simplest way to run a batch job on Windows is to open a command prompt window or the Start->Run dialog box and enter "sas -sysin mypgm.sas". You can also drop the icon for your program file onto the SAS icon.

Complete instructions for running SAS in batch on Windows are available in the SAS Companion for Microsoft Windows. If you don't have a copy, all the doc for SAS 9.1.3 is available here: http://support.sas.com/onlinedoc/913/docMainpage.jsp.

Re: Keeping the Log Window Clear

deleted_user — Wed, 15 Aug 2007 13:03:57 GMT

Yet another avenue to consider is the use of 'OPTIONS NONOTES;'. Obviously you wouldn't want this set when testing, but you shouldn't really be testing on this volume of data anyway.

Re: Keeping the Log Window Clear

advoss — Wed, 15 Aug 2007 13:29:22 GMT

Here is an example of code that I put in a batch file that allows a little more control over your environment and output. Please note, the line that invokes SAS shows up as about three lines in the post.

@echo off
time /T
if x%1==x? goto help
if /i x%1==xhelp goto help

REM figure out what last month (year and month) are
setlocal
for /f "usebackq tokens=1,2,3,4 delims=/ " %%i in (`date /t`) do set CCYYMM=%%l%%j
set /a xx=%CCYYMM% "%%" 100
if %xx% EQU 1 (set /a XCCYYMM=%CCYYMM% - 89) else set /a XCCYYMM=%CCYYMM% - 1
@echo %XCCYYMM%

"C:\Program Files\SAS\SAS 9.1\sas.exe" "H:\My_Documents\advoss\My_Master.sas" -autoexec h:\sas\autoexec.sas -sasuser h:\sas\sasuser -config "C:\Program Files\SAS\SAS 9.1\sasv9.cfg" -log "C:\Temp\My_Master_%XCCYYMM%.log" -noprint -nosplash
start notepad "C:\Temp\My_Master_%XCCYYMM%.log"
goto exit
:help
echo %0 will run My_Master SAS report
echo Invoke as %0
goto exit
:exit
time /T
echo on

Re: Keeping the Log Window Clear

deleted_user — Fri, 17 Aug 2007 07:07:47 GMT

Cheers All,

Using PRINTTO seems to have done the trick. Excellent stuff.

Re: Keeping the Log Window Clear

deleted_user — Fri, 17 Aug 2007 13:22:28 GMT

Hi

when it gets too large or unwieldy to maintain, you might like to reduce your command line that starts SAS by removing all the extra options (apart from -sysin) into a "batch" sasv9.cfg config file in folder of programs to be run in batch. That might reduce the command line to just

"C:\Program Files\SAS\SAS 9.1\sas.exe" "H:\My_Documents\advoss\My_Master.sas"

In that "batch" sasv9.cfg config file [pre]
-autoexec h:\sas\autoexec.sas
-sasuser h:\sas\sasuser
-RsasUser
-log "C:\Temp\My_Master_%XCCYYMM%.log"
-nosplash
-noterminal
-batch
[/pre]

when SAS starts, it will use the sasv9.cfg file next to the sas.exe, and one (when present) in the folder of the -sysin file.
You might also like the features of the -logparm option when running SAS in batch, it enables some #directives to put date and time into the log file name.
However, I prefer to use the name of the invoked program and it's not obvious how in this context.

PeterC

Re: Keeping the Log Window Clear

deleted_user — Fri, 17 Aug 2007 22:35:17 GMT

To clear the log at each iteration use something like:
dm log 'clear';

Re: Keeping the Log Window Clear

deleted_user — Mon, 20 Aug 2007 23:36:39 GMT

Peter's advice is, as always, very good. My preference is to retain a single CONFIG file because these are modified by versions of SAS, or even reinstallations on other PCs. I use alternative autoexec files for different purposes, autoexec for generic use, autobatch for generic batch etc.

However, writing logs off to a file where they are never read because they are too big is often used as a "cop out", and if you redirect your log, then I believe you need to make sure that the contents are still analysed in a timely manner. Very often I have seen logs redirected, never reviewed and a long period of incorrect information pass before someone goes to investigate some anomaly in the report.

It is better than turning off Notes with the NoNotes option, but not by much. I take an approach like the following. (I am writing this from memory, so it isn't tested.)

FileName INCLLOG Temp;

Proc PrintTo Log = INCLLOG;
Run;

Proc PrintTo Log = LOG;
Run;

Data _NULL_;
InFile INCLLOG TruncOver;
Input STRING $Char132.;
If STRING Eq: "ERROR" Then Put STRING;
Else If STRING Eq: "WARNING" Then Put STRING;
Run;

FileName INCLLOG Clear;

I add other statements to the log to select NOTES that are not expected by scanning for STRING Eq: "NOTES" and not containing key phrases for data set creation and so on.

Then I have a single log that tells me things I need to know, and when I have to process 600 text files as I did over the weekend I can surface the exceptions to my log and refine the log parse code and the exception reporting.

A block of code like this can then be held as a standard piece for inclusion in later programs of this type.

Kind regards

David

Re: Keeping the Log Window Clear

deleted_user — Tue, 21 Aug 2007 03:46:59 GMT

Realizing my specification of important NOTES messages may be vague, very good information is available from the SUGI paper presented by Delwiche and Slaughter entitled "Errors, Warnings and Notes: Oh my".

It is available on the SAS support site.

Kind regards

David

Re: Keeping the Log Window Clear

deleted_user — Tue, 21 Aug 2007 08:06:42 GMT

I don't like disagreeing with dkvj but if it appears I am on this occasion, it may only be caused by misunderstanding.
I recommend never updating the installed SAS configuration file. The administration or configuration of SAS System installation, provides features that support separating user- and application- specific options from the "platform-install" features. This separation is there by default. It is just unusual to find it because most expertise in this area developed before the "separation" became available in SAS8. In full support of customer legacy, most shrink-wrapped SAS installations invoke a single specific configuration file. Despite this, the separation can still be implemented with a feature demonstrated at the first SAS Global Forum in a paper entitled "Easy SAS® Session Customization" available at http://www2.sas.com/proceedings/forum2007/051-2007.pdf

David, you point out the reason for not using a single config file. That requires you to modify the SAS installed config file or even a copy of that, which may well be modified on the next update of the SAS System. My proposal leaves that file alone, keeping only customer options in a separate file.

I hope this "tirade" is helpful.

PeterC

Re: Keeping the Log Window Clear

deleted_user — Wed, 22 Aug 2007 03:48:33 GMT

In all the years we have known each other, I don't think we have ever disagreed Peter, just debated alternative points of view. I suspect the same is true now. I read your advice as suggesting the use of a config file specific to a purpose and I replied "retain a single CONFIG file because these are modified by versions of SAS, or even reinstallations on other PCs. I use alternative autoexec files for different purposes, autoexec for generic use, autobatch for generic batch etc.". So it seems we have the same reticence about changing the config.

However, I do change it for each installation, and all my version icons have paths to customised configs where I have set up my preferences for email, MsgLevel, FmtSearch and so on. I must also confess I modify below the line as well, by extending the Macro search path to include the macro libraries I have created for my or for group use. I then deploy the changed file to a SASWORK directory with other files available to one or all versions of SAS running on the machine.

Nonetheless, I define this change as "A THING TO DO VERY CAREFULLY" and to always retain a backup in case you overwrite the original. I noted for instance while updating a reinstall that the META-ID had changed from one install to another, so I carefully grafted my extra pieces on the new config and tested it.

However, please note I said that it is a config for a version, not for a task. I took your reply as suggesting configs could be at task level, and if I understood that correctly, then I disagree with that. I feel you might create AUTOEXEC, AUTOEXAF, AUTOBTCH and so on for different ways of using SAS and include application specific statements such as libraries and execution statements in the appropriate autoexec. I don't extend MgLevel and the other config changes into the autoexec because I think it makes them unweildy, and some of the changes I make are SAS invocation options, which can't be specified after the config.

As far as possible, IMHO, a config should contain those commands needed for every execution of that version of SAS, and adding options above the line should usually be safe. Then you do only have one config file for a SAS version.

Kind regards

David

Re: Keeping the Log Window Clear

deleted_user — Wed, 22 Aug 2007 10:03:56 GMT

my proposal is at application or project level, rather than by task.
The translation from these words to implementation:
a task means a .SAS program;
for a project/application, the SAS programs (tasks) to be invoked, reside in the same folder.
Also in that folder is a sasv9.cfg file. It is in this file that application-specific options are defined.
My proposal is to have an environment variable which suggests looking (also) in the "current folder" for a config file.

In this way the config file created as SAS is installed, can remain "untouched".
All your project-based options are stored in the project-folder.
In this way you can provide project specific information like
. format search paths,
. additions to the sasautos (discover SAS options -insert and -append)
. noOVP
. noterminal when appropriate
. sasuser pointer
. rSASuser (allowing share of sasuser features for concurrent batch work)
. set environment variables to use as libnames and filenames
. awsTitle to name the task to windows (something similar on unix?)
. lognote1 to provide support contact info in the saslog for maintainers
. termstmt to invoke program to send email indicating completion
. config to call another config file, e.g. for company-wide common options

I'm sure we can add to this list of project-specific options.

By making the environment variable pointer to a config file, non path-specific, the server can support multiple projects, each with its own special features, defined in the project-specific folder.

The same technique works well for a user environment. The "current folder" holds a config file with options specific to SAS programs in that folder and/or specific to the user's preferences.

Having a place for project-specific options, I try to put as little as possible in the autoexec.sas file. My preference/recommendation would be to use the -NOautoexec option for batch work.

Of course the invocation of a SAS session has an even more sophisticated routine for obtaining configuration files when none are defined on the sas.exe command line. However, I think you'll find my proposal is entirely consistent with that scenario.

Regards to the Forum

PeterC