topic Re: What's the maximum file size of a SAS dataset on a Unix server? in SAS Enterprise Guide

What's the maximum file size of a SAS dataset on a Unix server?

alepage — Fri, 15 Nov 2024 23:16:44 GMT

Hello,

I would like to know What's the maximum file size of a SAS dataset on a Unix server? I am receiving into my log file this error.

NOTE: Appending WORK.TABLE_INFO to WORK.FOUND_AGREEMENTS.
ERROR: Insufficient space in file WORK.FOUND_AGREEMENTS.DATA.
NOTE: There were 16720 observations read from the data set WORK.TABLE_INFO.
NOTE: 16719 observations added.
NOTE: The data set WORK.FOUND_AGREEMENTS has 56234190 observations and 23 variables.
ERROR: File WORK.FOUND_AGREEMENTS.DATA is damaged.
NOTE: Statements not processed because of errors noted above.
NOTE: PROCEDURE APPEND used (Total process time):
real time 2.15 seconds
cpu time 1.97 seconds

If I switch from base to spde engine, will it be better.

Which solution can I use ?

Re: What's the maximum file size of a SAS dataset on a Unix server?

ballardw — Fri, 15 Nov 2024 23:33:06 GMT

I suggest checking with your SAS admin about the settings for your WORK library. I think you are more likely to run into an issue with admin settings limiting file space then the size of a data set.

Re: What's the maximum file size of a SAS dataset on a Unix server?

Patrick — Sat, 16 Nov 2024 00:00:44 GMT

There must be some quota on your WORK area.

If you're one of these people who keep their SAS session alive "forever" then it's eventually just about some housekeeping. You could just start a new SAS session or you could within your existing session add a step that removes no more needed tables in work. Some code similar to below.

proc sql noprint;
  select memname into :memlist separated by ' '
  from dictionary.tables
  where libname='WORK' and memname not in ('TABLE_INFO','FOUND_AGREEMENTS')
  ;
quit;
proc datasets lib=work nolist nowarn;
  delete &memlist;
quit;

Re: What's the maximum file size of a SAS dataset on a Unix server?

alepage — Sat, 16 Nov 2024 00:11:10 GMT

I have reviewed my SAS code and I found that I was reading spde files and appending those into a base file. Could it be that ?

I have set libname dest1 spde "&path1". and appending base=dest1.found agreement . I hope it will works

Re: What's the maximum file size of a SAS dataset on a Unix server?

alepage — Sat, 16 Nov 2024 00:28:07 GMT

does the drop statement will do the same:

proc sql;
drop table table_info;
quit;

Re: What's the maximum file size of a SAS dataset on a Unix server?

Patrick — Sat, 16 Nov 2024 03:18:22 GMT

@alepage wrote:
does the drop statement will do the same:

proc sql;
drop table table_info;
quit;

It does but you can only drop one table at a time and the table must exist.

With Proc Datasets NOWARN you can drop multiple tables at once and the command will also work should a table not exist.

Re: What's the maximum file size of a SAS dataset on a Unix server?

Ksharp — Sat, 16 Nov 2024 06:33:04 GMT

As Patrick and ballardw said your account has limited quota of disk storage , talk to your UNIX Admin to unlease your quote by command :
xfs_quota -x -c 'limit -u bsoft=80M bhard=100M username' /data

P.S: use command
quota username
to chek your quota.