BookmarkSubscribeRSS Feed
vapodaca11
Calcite | Level 5
I am working with large datasets - 5TB a piece.

When I run a sort procedure I save them to the actual space which is over 100TB, but for some reason they are running through the tmp folders which only has 1TB so the sort execution fails. How do I get the dataset to sort in the folder space and not the tmp space to keep if from failing.

So by folder space I mean it matches up with a Libame, so mc.dataset
3 REPLIES 3
Kurt_Bremser
Super User

Temporary utility files are created in the WORK location, or in the location specified by the UTILLOC option. You need to change these options in the configuration if you want to have your WORK and/or UTILLOC somewhere else than /tmp.

Which configuration you need to change depends on the way you use SAS (Base SAS with Display Manager, Enterprise Guide with Local server, remote workspace server with EG or SAS Studio, SAS in batch mode).

Kurt_Bremser
Super User

One thing you can try is to sort the dataset in a proc sort step, and use the tagsort option. This means that only the by variables and an observation pointer go into the utility file, but you pay for that in terms of performance.

Tagsort is also good when you have used the compress=yes option on a dataset and managed to get a high compression rate.

Ksharp
Super User

This table is too big for PROC SORT or SQL .

I would suggest other method to sort data:

 

data F M;

 set sashelp.class;

 select(sex);

 when('F') output F;

 when('M') output M;

 otherwise;

run;

 

data want;

 set F M;

run;

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 3 replies
  • 1265 views
  • 0 likes
  • 3 in conversation