topic Re: Proc Sort Command Optimisation in New SAS User

Proc Sort Command Optimisation

ssaha — Tue, 26 Feb 2019 10:10:48 GMT

When I am using Proc Sort in a SAS program, our work directory space in Unix Server is increasing continuously and it is getting halted at some stage due to memory issue.

How we can optimise the Proc Sort command so that the work directory space doesn't increase much.

Re: Proc Sort Command Optimisation

RW9 — Tue, 26 Feb 2019 10:24:36 GMT

How big is the data you are talking about? If its reasonable size and still filling your storage space, then get a bigger storage space. If its huge data then you might need a whole new infrastructure.

As for optimising your code, without seeing any code, knowing anything about your process or system, or knowing anything about your data, I don't see how it is possible to provide any useful information. Why do you need to sort for instance? Can the data not be sorted up front. What about using an index? How is the sort to be used further along. How is the data stored, is it locally, network, database etc.

Re: Proc Sort Command Optimisation

ballardw — Thu, 28 Feb 2019 18:25:19 GMT

@ssaha wrote:

When I am using Proc Sort in a SAS program, our work directory space in Unix Server is increasing continuously and it is getting halted at some stage due to memory issue.

How we can optimise the Proc Sort command so that the work directory space doesn't increase much.

Are you using the option TAGSORT on you Proc Sort statement? That can significantly reduce the temporary data disk space required.

From the documentation:

When the total length of BY variables is small compared with the record length, TAGSORT reduces temporary disk usage considerably. However, processing time might be much higher

Re: Proc Sort Command Optimisation

Astounding — Thu, 28 Feb 2019 18:34:12 GMT

Perhaps you should work on the data first. Does it contain variables that are $200 characters long, but only really need 5 characters of storage? Do you know how to examine that and how to fix it?