BookmarkSubscribeRSS Feed
vapodaca11
Calcite | Level 5
I am working with large datasets - 5TB a piece.

When I run a sort procedure I save them to the actual space which is over 100TB, but for some reason they are running through the tmp folders which only has 1TB so the sort execution fails. How do I get the dataset to sort in the folder space and not the tmp space to keep if from failing.

So by folder space I mean it matches up with a Libame, so mc.dataset
3 REPLIES 3
Kurt_Bremser
Super User

Temporary utility files are created in the WORK location, or in the location specified by the UTILLOC option. You need to change these options in the configuration if you want to have your WORK and/or UTILLOC somewhere else than /tmp.

Which configuration you need to change depends on the way you use SAS (Base SAS with Display Manager, Enterprise Guide with Local server, remote workspace server with EG or SAS Studio, SAS in batch mode).

Kurt_Bremser
Super User

One thing you can try is to sort the dataset in a proc sort step, and use the tagsort option. This means that only the by variables and an observation pointer go into the utility file, but you pay for that in terms of performance.

Tagsort is also good when you have used the compress=yes option on a dataset and managed to get a high compression rate.

Ksharp
Super User

This table is too big for PROC SORT or SQL .

I would suggest other method to sort data:

 

data F M;

 set sashelp.class;

 select(sex);

 when('F') output F;

 when('M') output M;

 otherwise;

run;

 

data want;

 set F M;

run;

SAS Innovate 2025: Save the Date

 SAS Innovate 2025 is scheduled for May 6-9 in Orlando, FL. Sign up to be first to learn about the agenda and registration!

Save the date!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

SAS Training: Just a Click Away

 Ready to level-up your skills? Choose your own adventure.

Browse our catalog!

Discussion stats
  • 3 replies
  • 1477 views
  • 0 likes
  • 3 in conversation