Disk Space issue

[Ranjeeta]

data NAC_FY1718_emerg_1; set NAC_FY1718_emerg;
261 Facility_No_New=put(Facility_No, $3.);
262 drop Facility_No;
263 rename Facility_No_New=Facility_No;
264 run;

ERROR: Insufficient space in file WORK.NAC_FY1718_EMERG_1.DATA.
NOTE: The DATA step has been abnormally terminated.
NOTE: There were 4460704 observations read from the data set WORK.NAC_FY1718_EMERG.
WARNING: The data set WORK.NAC_FY1718_EMERG_1 may be incomplete. When this step was stopped there were 4460704 observations and 376 variables.
NOTE: DATA statement used (Total process time):
real time 1:38.98
cpu time 19.09 seconds

Hello After this step if i run any other data steps there is nothing printed in the log

Can you advise why?

Thanks

[ChrisNZ]

>After this step if i run any other data steps there is nothing printed in the log Can you advise why?

The log should still be there, but your WORK directory is full at this point, so you can't do much at all.

Delete unwanted tables as you go to be able to run your process.

[hashman]

@Ranjeeta:

If you keep coding like that, you'll run out of disk WORK space no matter how much of it you have allotted. 

To wit, you already have a huge file, and yet you are creating an almost exact copy of it, just as huge, without adding any new information to it. In fact, you're losing some, for all your step does is keeping the 3 leftmost bytes of Facility_No and discarding the rest. You can do it in the ensuing processing on the fly without any need to create the new huge file.

Alternatively, if you really need Facility_No to be $3 instead of whatever it currently is, create a view, for example:  

data NAC_FY1718_emerg_1 / view = NAC_FY1718_emerg_1 ;
  set NAC_FY1718_emerg (rename=(Facility_No=_F)) ;   
  Facility_No = put (_F, $3.) ;                      
  drop _F ;                                          
run ;                                                

Then in the ensuing processing, just read this view. It takes virtually no disk space, so you'll be much safer on that head.

Kind regards

Paul D. 

[Ranjeeta]

Thankyou that's a very helpful tip
Can views be used in a sas session similar to a dataset that was saved or there are some obvious restrictions

Regards

[Kurt_Bremser]

---

@Ranjeeta wrote:
Thankyou that's a very helpful tip
Can views be used in a sas session similar to a dataset that was saved or there are some obvious restrictions

Regards

---

Views can be used as input anywhere where you use a dataset. The logic contained in the view is executed when the view is used.

Note that SAS has two type of views: data step views (what @hashman used in his example) and SQL views. A data step view that combines datasets in a merge does not need additional disk space while it's executing, but a SQL view with a join, group by or order by will still create utility files.

[hashman]

@Ranjeeta:

A view (actually, its definition) is saved in the WORK library if you qualify it with a 1-level name. Then for the duration of the session, it is good for input as long as the underlying data set(s) they reference are not deleted or not modified to the degree that a view cannot use them (for example, you've modified a data set referenced in a view and dropped a variable the view references). A view saved to WORK exists for the duration of the SAS session, just like a data set (unless the NOWORKTERM system option is in effect). You can also save a view in a permanent library, but then again, at the time it is used, the data sets(s) it references must be in place with all the requisite attributes the view requires. 

Generally speaking, views (both DATA step and SQL) are very powerful means of improving SAS performance; one just have to give a decent shot at learning how they work to use them productively. Of course, they involve some behind-the-scenes overhead in terms of so-called spill files; but it's nothing compared to making a physical copy of a huge file just for the sake of creating a new variable or renaming an existing one. 

Kind regards

Paul D.

[Kurt_Bremser]

Just to give you a real-life example what views can be good for:

In 2002, Austria converted from the old currency (Schiliing) to the Euro, at a conversion rate of 13.7603 ATS = 1 EUR.

At the same time, all our contracts (I work for an insurance company) were converted, but we had to keep lots of SAS datasets from previous periods. In order to keep the historic datasets as-is, we renamed them to _[old dataset name] and created views from them (with the old dataset names) where all amount columns were multiplied with the conversion factor. This allowed existing programs to work seamlessly with the new data, and timeline comparisons will give meaningful results.

[Kurt_Bremser]

Maxim 3: Know Your Data.

If there are (long) character variables, using the compress=yes dataset option can reduce disk space consumption considerably.

You are also showing just the last failing step (the inefficiency of which was already pointed out by @hashman), but I guess your code further upstream contributes greatly to your disk space issues. By showing us more of the code, you could help us a lot in helping you.

When preparing data for analysis, reduce as early as possible. Drop unneeded columns (or keep the needed ones), and use where or subsetting if to reduce the number of observations, if such is possible by your final logic.