Batchscript fails when used via putty but succesfull concludes when run via Enterprise Guide

[AIS1]

Hello,

we have a relatively weird problem.

Using SAS Enterprise Guide 7.1 and SAS 9.4.

We have a datamart, which we used to rebuild daily using an automated process called "build_data_mart".
This script worked fine for a long time, but suddenly stopped being succesfull about a week ago.

Whenever this script is run automatically, it runs into the following error in the appropriate log files:
File WORK.'SASTMP-000000028'n.UTILITY is damaged. I/O processing did not complete

It also runs into this error at effectively the same step.

When I activate the script via putty, I receive the following message:

OUT OF RESOURCES  Select:
  R. Retry
  N. Tell procedure/DATA step no more resources.
  C. Cancel Submitted Statements
  T. Terminate SAS.
?

Retry does nothing, if I cancel, the resulting log does not show an error but ends in the same spot of the SQL code.

Interestingly enough, if I execute the process via the Enterprise Guide, it runs without any problems whatsorever.

There are ample amounts of disc space available in the work directory, the same appears to be true for physical memory.

We have also verified that there is not quota allocation.

The step at which it fails is nothing particularly serious. The resulting table has, if it is run via Enterprise Guide, 28 columns (yes, we kind of need them) and some 489K observations, resulting in 282 pages compressed and 783 pages uncompressed. While not neglible, this is not a terribly large table, other tables the script creates before are larger.

Questions: Is there a way to find out with which options SAS Enterprise guide runs the script? How can I replicate these options for the batch script?

[RW9]

You tend to see that message when the printed file reaches a certain level, i.e. procedures which print lots of output to to listing destination. 

Try putting ods _all_ close; before your code.  Also put noprints on procedures you don't need listing output.  Turning off other reporting output will also likely help, i.e. options nonotes nosymbolgen nomprint mmlogic; etc.

Basically minimizing listing output.

[VDD]

this is one way I keep my log clear.  

*-------------------------------------------------*
* Clear Results Viewer
*-------------------------------------------------*;
ods html close; 
DM log "OUT;CLEAR;LOG;CLEAR;" log continue ;
DM log 'next results; clear; cancel;' whostedit continue ;
ods html newfile=none;

[AIS1]

Hi,

I have put an " ods _all_ close ;  "

before the entire statement, there does not seem to be much of an effect according to the log.
I can see the ods_all_close statemnet in the log, but I do not see the option doing anything.

Did I put it in the wrong place?

The whole file is just something like

%let xyz_basedir = /sas/sasdata/sascode/sas_sale;

ods _all_ close ;

%include "&xyz_basedir./tsr_autoexec.sas";

%include "&xyz_basedir./build_data_mart.sas";

Should the odds_all_close be in the respective subtasks instead?

Code ran out of resources as before.

[VDD]

As @RW9 wrote put noprints on procedures

[RW9]

Its nearly impossible for us to say without seeing something.  Run your code interactively and see what is being produced in the ods destinations, a simple stats procedure or loop could produce thousands of pages of text output to ods, which would be removed by setting the procedure to noprint.  ods _all_ close; however should turn off all ods destinations.  You will need to do some debugging, maybe an include is turning on an ods destination or something.

[Kurt_Bremser]

Is the user used in putty and EG the same?

Do you have a UNIX or Windows server?

There are some peculiarities with processes spawned from the Object Spawner, as they inherit settings (eg memory limits) from the spawner, and therefore from the SAS installation user (if you used defaults during installation), while batch processes inherit those settings from the user running them.

If your spawner is running under, say (UNIX example)

ulimit -d unlimited

then your EG workspace server process enjoys an unlimited environment, while logging on with the same user via putty will get you the limits defined for that user in the system base.

[AIS1]

Hi Kurt,

I am reasonably certain, after consulting the logs, that the user may not be the same. In addition, the failed logs have the following:

NOTE: Unable to open SASUSER.PROFILE. WORK.PROFILE will be opened instead.

I could not actually find the *.Profile files anywhere in the SAS directory though.

Will try executing file as the same user the enterprise guide has.

I am also trying the ulimit -d unlimited option in front of the batch script.

[Kurt_Bremser]

The ulimit -d was just an example (it concerns memory settings).

SASUSER.PROFILE points to a catalog (~/sasuser.v94/profile.sas7bcat) that SAS needs to have exclusive (because of updates) access to; anytime more than one process runs for the same user, you get that message. It's uncritical, because it's mostly for settings of the interactive (Display Manager) environment.

Your EG user might have a different quota than the one you use from the batch. Or the batch might have a different location for WORK, because it uses a different configuration file.

If you use SASApp in EG, use (sas conf directory)/Lev1/SASApp/BatchServer/sasbatch.sh to call SAS in your batch jobs; it provides you the same basic environment that your workspace server has.