BookmarkSubscribeRSS Feed
derrick
Calcite | Level 5

Running SAS 9.2 on Windows.

I have enabled MEMCACHE and MEMLIB option.

My program does not use any large data files.  The largest dataset is about 500k.  75% of the time the error is related to the OUT= dataset form PROC SORT.  The other 25% of the time, no dataset is named and I assume the error is coming from PROC IML.  I tried using SORTSIZE= and NOTHREAD options in the SORT procedure as well as increasing memory allocated to PROC IML to 2GB each in the SYMSIZE and WORKSIZE options.   I have no idea how to pinpoint the source of the problem.

ERROR: Utility file open failed.

ERROR: Open of file WORK._tf0001.UTILITY for OUTPUT failed; another copy of the file is open for

       OUTPUT/UPDATE.

WARNING: The data set WORK.SORTHAZ3 may be incomplete.  When this step was stopped there were 0

         observations and 4 variables.

WARNING: Data set WORK.SORTHAZ3 was not replaced because this step was stopped.

ERROR: SAS ended due to errors.

       You specified: OPTIONS ERRORABEND;.

9 REPLIES 9
derrick
Calcite | Level 5

Is there any way to get at how large the utility file is?  When I turn off the memlib option and browse to the work directory I don't see any files in the utility directory when the program is running.

AncaTilea
Pyrite | Level 9

When you say utility directory, do you mean your WORk library? Or USER library?

Are you basically running a

PROC SORT DATA = your_data OUT = problem_data;RUN;

?

I think things may be over my head from here on...

:smileyconfused:


derrick
Calcite | Level 5

Within the current WORK directory there are two temp directories. One named _TDXXXXX and the other named SAS_utilXXXXXXX.  The utility one never has any files in it.

AncaTilea
Pyrite | Level 9

Here is another link that may be helpful (even though it relates to UNIX env)

(but it may applied to you as well)

http://support.sas.com/kb/9/238.html

jakarman
Barite | Level 11

As you stated: Without having set the Memcache Memlib options the program is running fine.

The problem you posted may be caused by one or both (combination) of these options.
As the they are tuning options to move io/data to memory you could find other solutions to do that.

The root cause of this type of errors can be a result of OS (Windows) behavior and/or settings. I had some bad experiences with SGIO option, never solved just switched off. When you need to analyze and solve that part you should go for the TS support approach.  It would be interesting to see some feedback of that.

Although you have small datasets (0,5Mb) you could have run out of memory.

I am missing the analyses part on that.

The error is indicating a locking problem on an internal processed file. That is weird.

Soemtimes this happens by very advanced tuning ideas to assume have delayed processing and holding files open (opportunistic locking). It is at the edge of OS level internals and  SAS.  

You have found the Work location. the _TDxxxx (xx are random assigned processnumbers) you will find work files as you could see by the SASwork. In the current SAS-session these locations must be the same.

The SAS_utill ....  Your error-message  is indicating is using WORK, SASwork to be at that physical location.

File processing is often cached at OS level. Juust at some synchronisation points it will write some information out.

By that you can expect not seeing anything happening and  .... at the same time the OS is very busy working on that.
The more advanced trace/debug and monitor tools should be able to help and debug that kind of activity. 
   

---->-- ja karman --<-----
derrick
Calcite | Level 5

What I'm hearing is it is probably due to the OS doing some background task like a virus scan and locking the file.  The temp directories in the WORK directory are created but they are always empty, which is consistent with the MEMLIB option.  Turning off MEMLIB and MEMCACHE seems to be the way to go.

jakarman
Barite | Level 11

SAS(R) 9.4 Companion for Windows (Performance tools)

---->-- ja karman --<-----
Peter_L
Quartz | Level 8

I have had this regularly too on 9.2 TS2M3 Windows. It attacks a few specific jobs which tend to be long-running or with larger data sets. It is reproducible for those jobs. I have only MEMLIB set. I have messed about with various settings and options to no effect. MEMLIB is necessary to protect my files from Trend Micro virus scanner, which locks the files which causes SAS to fail. I just tell my users that it is a design fault in SAS. I know I could ask tech support for help but in my experience this type of error is a black hole for my time.

 

The various suggestions in this thread are either irrelevant for Windows or do not work. I have tried them.

 

Why can't we have an option in SAS to make it live with anti-virus software, e.g. by setting a wait or retry time before it just throws in the towel and spits out an error message? No other software I use is intolerant of anti-virus software, database included. Ah, I was forgetting Windows itself there ;-).

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 9 replies
  • 6708 views
  • 0 likes
  • 4 in conversation