topic What is the tip for shrink SASdata in SAS Health and Life Sciences

What is the tip for shrink SASdata

bbb_NG — Thu, 16 Feb 2012 06:04:33 GMT

Hi,

I have a sas7dbat file

Name	Type	Length	Format	Informat	Label	Comments
branch_code	Character	7	$CHAR7.	$CHAR7.	BRANCH CODE	All value are of three english character
PFS Cust Segment Code	Character	6				All value are of three english character
LoanCMP	Character	1
OutstandingCMP	Character	1
NationCMP	Character	3
Verdate	Character	204				All value are of 6 numberic character as "201112"
New Cust Ind	Character	2	$2.00	$2.00	New Cust Ind	All value are of one english character
OccupationCMP	Character	13				The longest value for this field is 10 english characters.
cmp	Character	25				The longest value for this field is 10 english characters.
NoCount	Numeric	8				??it's length limit was 8, but I do find a value over 100M
TRBLevel	Character	9

with 3.2M records and 837M disk space.

How to shrink it?

Thanks in advance.

What is the tip for shrink SASdata

art297 — Thu, 16 Feb 2012 13:08:13 GMT

This sounds like a quiz question. Since unnecessarily used space still takes up space, you are wasting a lot of space in all of the character fields. As for the numeric one, read the documentation: http://support.sas.com/documentation/cdl/en/hostunx/61879/HTML/default/viewer.htm#a000344718.htm

What is the tip for shrink SASdata

Hima — Thu, 16 Feb 2012 14:17:31 GMT

Zipping the data set might help.

What is the tip for shrink SASdata

Astounding — Thu, 16 Feb 2012 14:42:14 GMT

First, you have to be very sure that your "Comments" are correct. For example, you have to be 100% certain that VERDATE is never longer than six characters. Since BRANCH_CODE uses a $CHAR7. format/informat, it is not enough to print some values. The $CHAR formats imply that the variable might contain leading blanks. You have to determine if it does contain leading blanks, whether you would like to left-hand-justify the characters. The length needed might be impacted, but the sorted order might change as well.

After you have completed your analysis, set the option and the lengths appropriately. For example:

options compess=YES;

data my_data;

length VERDATE $ 6;

set my_data;

run;

Set the length BEFORE the SET statement, for as many variables as you have analyzed to verify the needed length.

Good luck.

What is the tip for shrink SASdata

Tom — Thu, 16 Feb 2012 15:36:16 GMT

You will probably get pretty good reduction by just using the COMPRESS=YES option.

If you are correct about the maximum lengths of the actual data then you can make the file smaller by re-creating it with shorter character variables.

Where you would want to avoid this is if this is just one example of a number of datasets that you have with this structure. The other potential datasets might have longer actual values.

There is usually not much value (and some risk) in storing numbers in less than 8 bytes. SAS always stores numbers are floating point.

The easiest way to recreate with shorter lengths is to set the length before referencing the original data (as suggested above).

Watch out for the variables that have permanent formats attached. You should change those (or better just remove them). You will need to put the FORMAT statement after the set statement for it to win out.

data want ;

length branch_code $3 .... ;

set old ;

format _character_ ;

run;

What is the tip for shrink SASdata

bbb_NG — Fri, 17 Feb 2012 02:21:21 GMT

Dear All,

Thank you very much for your help and sharing thoughts.