topic Re: Reg:TO find the Size (Memory) of The New Table in SAS Procedures

Reg:TO find the Size (Memory) of The New Table

R_Win — Mon, 17 Jan 2011 10:20:20 GMT

Hi i am having two tables and i should join them by using proc sql and the joining columns are unique and the size of each dataset is 18-20 gb can u let me know is there any chance to fine the size of the new dataset ,Prior to join as there was less space i my memory

Re: Reg:TO find the Size (Memory) of The New Table

SteveFrensch — Mon, 17 Jan 2011 13:35:49 GMT

Cant say forsure on the size issue, but if you are running into space issues, make sure you are at least using some form of compression:

options compress=binary;

Re: Reg:TO find the Size (Memory) of The New Table

sbb — Mon, 17 Jan 2011 14:53:48 GMT

Suggest starting with PROC CONTENTS output - also, refer to details in your particular OS environment's SAS "companion" documentation.

Scott Barry
SBBWorks, Inc.

Suggested Google advanced search argument, this topic / post:

proc contents estimate file size site:sas.com

Re: Reg:TO find the Size (Memory) of The New Table

Doc_Duke — Mon, 17 Jan 2011 16:25:53 GMT

You can roughly compute the resulting data set size with the

("length of record 1" + "length of record 2") * (NObs1 + Nobs2 - NMatchingObs)

You could get closer with two test runs. One with all the row and limit to the joining columns (to find the total number of observations) and one with just a few rows from each table (to find the length of one row).

However, if you are having space difficulties, you also need to look at the amount of work space that SQL will use. For that, you need the SAS companion for the OS and the SAS manual for SQL. If you are on a mainframe, SQL may use an external sort procedure depending on what your local admins set up.

Personally, I find the OS compression to be superior to SAS compression if that is available to you.

Doc Muhlbaier
Duke