10-29-2015 02:10 PM
This is a technical question regarding to what occurs during a set creation-
If you are already creating a new dataset from an old dataset and you have some added variables or some other changes can the dataset sort of balloon or expand during its creation and then upon it finishing-then applies compression (if option is set) ?
And can this happen during a data step and/or sql procedure?
The reason I ask is that last week-someone's data step suddenly hit the space constraints on a unix server.
Although the dataset was only 20 gigs with 30 gigs of space left it was pretty evident it was the activity that led to some further space constraint issues.
10-29-2015 02:55 PM
When SAS is creating a new table the same name as an existing table, there will be two versions of this table kept until SAS successfully finishes creating the new version. The reason for this is so that SAS does not over-write the existing table if there is an error. Once the step is successful the old version table is deleted and the new version table is renamed to be the same as the old version one.
To summarise you need enough disk space to store two versions of the same table.
10-29-2015 11:21 PM
Then there wouldn't be two versions of the new table. However as soon as you try to overwrite the new table by running the same program twice, two versions will be kept. This applies for both permanent and temporary tables.
10-30-2015 11:44 AM
Some operations may even need more space.
eg sorting a data set will need triple space: the original file, the intermediate (utility) file and the new file (the utility file is either in WORK or where UTILLOC points to)
If the dataset is compressed, this will not affect the utility file, so you may need eben more space than just triple the original filesize.
SQL operations also may create huge utility files, depending on the type of operation.