SAS GRID - View log file (from proc printto) while batch program is running

[SASKiwi]

I have no problem at all opening logs of running SAS batch jobs under Windows. I don't buy the idea this isn't possible under linux. It is just a matter of finding the correct folder location and using the appropriate OS command. By default output to SAS logs is buffered so you don't necessarily see immediate processing. You can use this option to write to logs immediately:

sas -sysin "my-batch-program" -LOGPARM=“WRITE=IMMEDIATE”. This may impact performance though. 

[jwilson]

Hi SASKiwi,

Would it be possible for you to share the contents of your log config files located in the directory specified by the output of:

proc options option=logconfigloc value ; run ;

So I can compare with mine?

[Tom]

---

@jwilson wrote:
Hi SASKiwi,

Would it be possible for you to share the contents of your log config files located in the directory specified by the output of:

proc options option=logconfigloc value ; run ;

So I can compare with mine?

---

Never heard of that option. And apparently we are not using it. Even though we have been running SAS on GRID for years.

Option Value Information For SAS Option LOGCONFIGLOC
    Value:
    Scope: Default
    How option value set: Shipped Default

Looking at the help page for that option it does not appear to have anything to do with where the SAS log for a program goes.  It looks like it is for logging the status of the application server or workspace server perhaps.

[Kurt_Bremser]

Concur with the use of WRITE=IMMEDIATE. The performance impact is negligible.

[jwilson]

I tried the LOGPARM options and they did not work. The documentation mentions that this option only works in connection with ALTLOG. Though I also tried the ALTLOG option and it still did not work.

[Kurt_Bremser]

You are misreading the documentation because you miss an important detail:

Interaction This argument is valid in the windowing environment when SAS is started with the ALTLOG= system option. The contents are written to the file that is specified by the ALTLOG= option and to the SAS log.

This sentence is irrelevant for batch jobs.

[jwilson]

I guess either way it didn’t work. I tried every combination of these options and didn’t find any that made the log readable.

[Kurt_Bremser]

This has never been an issue with SAS 9.4 (and all previous versions). This is either caused by the grid filesystem or Viya. In both cases, SAS has to fix it or provide a workaround.

[Patrick]

@jwilson Did you already try and batch submit a job using sasbatch.sh as mentioned in my previous post? sasbatch.sh should get created as part of a SAS install.

@Kurt_Bremser What's the setup you've used to surface the logs via Web pages?

[Kurt_Bremser]

When you link to a .log file, it will automatically be served as is, as simple text.

The root directory of your logs tree needs to be defined in the Apache configuration, if it is not a subdirectory of DocumentRoot already.

I kept track of all batch jobs in a dataset (populated automatically from the sasbatch script), and from this created a webpage with a table of jobs, and that contained all the links to the log files.

[SASKiwi]

@jwilson  - What does the log look like when you view it? If it looks like line feeds are not being honoured then that could be a problem with your viewing tool. Notepad++ deals with this better than most - choose Edit Menu option EOL Conversion Unix (LF).

[Kurt_Bremser]

The fact that a simple (and long time proven!) utility like tail crashes with a segfault and coredump when trying to read a file points to some serious issue. If the file was not readable for some reason, tail would say so and fail gracefully.

[Kurt_Bremser]

I've never had an issue with reading log files while a batch job was running. "LIve" log files were available as web pages on my server, so I could follow a job's progress by refreshing the page.

[gwootton]

The SAS Grid Manager Client Utility (sasgsub) is used to submit jobs to the SAS Grid for execution. This makes use of the GridServer (<SASConfig>/Levn/SASApp/GridServer) to run the code. SASGSUB will store the log in a separate location during exeuction (the Grid work directory defined during installation) and has options for pulling the log or output either by waiting until the job completes (-GRIDWAITRESULTS) or as a separate command (SASGSUB GRIDGETRESULTS), which will the output/log to the current working directory. So, the options for monitoring the log as it is being written are to follow the log in the shared grid work location (this is output when the job is submitted under "Job log file"), or use ALTLOG to write to an alternate location simultaneously and watch it there. The format would be something like:
sasgsub -gridsubmitpgm /path/to/program.sas -gridsasopts "( -altlog /path/to/altlog.log)"

[sas@trcv003 9.4]$ pwd
/sas94/config/compute/Lev1/Applications/SASGridManagerClientUtility/9.4
[sas@trcv003 9.4]$ cat test2.sas
proc options; run;
[sas@trcv003 9.4]$ ./sasgsub -gridsubmitpgm test2.sas

SAS Grid Manager Client Utility Version 9.45 (build date: Oct 3 2017)
Copyright (C) 2009-2017, SAS Institute Inc., Cary, NC, USA. All Rights Reserved

Job <48401> is submitted to queue <priority>.
Job ID: 48401
Job directory: "/sas94/gridshared/sas/SASGSUB-2022-10-03_09.09.38.345_test2"
Job log file: "/sas94/gridshared/sas/SASGSUB-2022-10-03_09.09.38.345_test2/test2.log"
[sas@trcv003 9.4]$ ./sasgsub -gridsubmitpgm test2.sas -gridsasopts "(-altlog /sas94/gridshared/logdemo.log)"

SAS Grid Manager Client Utility Version 9.45 (build date: Oct 3 2017)
Copyright (C) 2009-2017, SAS Institute Inc., Cary, NC, USA. All Rights Reserved

Job <48402> is submitted to queue <priority>.
Job ID: 48402
Job directory: "/sas94/gridshared/sas/SASGSUB-2022-10-03_09.10.36.689_test2"
Job log file: "/sas94/gridshared/sas/SASGSUB-2022-10-03_09.10.36.689_test2/test2.log"

[sas@trcv003 9.4]$ head /sas94/gridshared/logdemo.log
1 The SAS System 09:10 Monday, October 3, 2022

NOTE: Copyright (c) 2016 by SAS Institute Inc., Cary, NC, USA.
NOTE: SAS (r) Proprietary Software 9.4 (TS1M5)
Licensed to 9.4 M5 LINA LSF Grid TRCV003 Renewal, Site 70068118.
NOTE: This session is executing on the Linux 3.10.0-693.2.2.el7.x86_64 (LIN X64) platform.



NOTE: Updated analytical products:
[sas@trcv003 9.4]$

[Tom]

So the sasgsub command is jumping through hoops to move log files around (I assume there is a reason for this, probably related to desire to avoid having to cross mount all of the filesystems involved).

But that does not explain why someone could not look at the file while it was being written.

or why they cannot look at the files they are explicitly creating somewhere else by using PROC PRINTTO.

Perhaps the trouble is related to this complex method of running the code.  Since the SAS code is running on a different machine than the one they submitted it on perhaps they cannot see the files being generated by using PROC PRINTTO because the location the code is writing to is not visible on the machine they used to submit the job.