BookmarkSubscribeRSS Feed
R_Win
Calcite | Level 5
Hi i am having two tables and i should join them by using proc sql and the joining columns are unique and the size of each dataset is 18-20 gb can u let me know is there any chance to fine the size of the new dataset ,Prior to join as there was less space i my memory
3 REPLIES 3
SteveFrensch
Calcite | Level 5
Cant say forsure on the size issue, but if you are running into space issues, make sure you are at least using some form of compression:

options compress=binary;
sbb
Lapis Lazuli | Level 10 sbb
Lapis Lazuli | Level 10
Suggest starting with PROC CONTENTS output - also, refer to details in your particular OS environment's SAS "companion" documentation.

Scott Barry
SBBWorks, Inc.

Suggested Google advanced search argument, this topic / post:

proc contents estimate file size site:sas.com
Doc_Duke
Rhodochrosite | Level 12
You can roughly compute the resulting data set size with the

("length of record 1" + "length of record 2") * (NObs1 + Nobs2 - NMatchingObs)

You could get closer with two test runs. One with all the row and limit to the joining columns (to find the total number of observations) and one with just a few rows from each table (to find the length of one row).

However, if you are having space difficulties, you also need to look at the amount of work space that SQL will use. For that, you need the SAS companion for the OS and the SAS manual for SQL. If you are on a mainframe, SQL may use an external sort procedure depending on what your local admins set up.

Personally, I find the OS compression to be superior to SAS compression if that is available to you.

Doc Muhlbaier
Duke

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 3 replies
  • 1075 views
  • 0 likes
  • 4 in conversation