topic Re: SAS Datasets size in SAS Procedures

SAS Datasets size

Tal — Mon, 17 Jul 2017 22:44:58 GMT

Hi,
My co-worker and I are running the same program on pc sas but remotly loginin in on 2 different SAS servers. The size of the datasets.my pc sas creates are twice the size hers does. How do you guys explains this. Anything i can change in the settings of my pc sas? Any idea please? Thx

Re: SAS Datasets size

Reeza — Mon, 17 Jul 2017 22:47:13 GMT

Run a PROC CONTENTS and look for differences. ie does she have the compress option set on by default.

Re: SAS Datasets size

Tal — Mon, 17 Jul 2017 23:28:34 GMT

By default you mean some settings in pc sas?

Re: SAS Datasets size

Tom — Mon, 17 Jul 2017 23:45:26 GMT

How did you check the size? What units did the size get reported in? Is it possible one is counting number of 512 byte blocks and the other is reporting number of Kilobytes?

Re: SAS Datasets size

ChrisBrooks — Mon, 17 Jul 2017 23:54:37 GMT

Whate Reeza is suggesting is that you both run this (substituting the name of your datasets for sashelp.class).

proc contents data=sashelp.class;
run;

Then check the output which should look something like this

I've circled the bit to check - if hers says "Yes" and yours says "No" that's the answer - you can change the default setting by putting this in your autoexec,sas file - you should be aware though that it's possible for small compressed files to be slightly larger than their uncompressed versions.

options compress=yes;

Re: SAS Datasets size

Tal — Tue, 18 Jul 2017 00:03:18 GMT

I go to the unix folders through a ftp. Hers shows 3.2GB mine for example 7.8GB

Re: SAS Datasets size

Tal — Tue, 18 Jul 2017 00:05:26 GMT

Chris we both use exactly the same program. If she has that global option compress=yes then I d have it too but i will still run proc contents.

Thx all for the advices

Re: SAS Datasets size

ChrisBrooks — Tue, 18 Jul 2017 00:10:36 GMT

The point is the options statement doesn't have to be in the program - autoexec.sas (if it's present in the correct location) will run at the start of every session and can set options for the whole session unless they are changed.

Re: SAS Datasets size

Tom — Tue, 18 Jul 2017 00:10:39 GMT

Depending on how your disks are configured on unix some will report the physical space used instead of the logic space used. The du command on linux now supports an --apparent-size option. I have some linux machines I am using now where there can be a large difference in the size reported depending on whether the --apparent-size option is used or not.

That said it is probably more likely that something else is different in how SAS ran the code. Could be some minor difference in the code, oe perhaps you actually are reading different source files, or something about the settings like the COMPRESS= option, or even the blocksize that SAS used to write the observations.

Re: SAS Datasets size

Tal — Tue, 18 Jul 2017 00:29:13 GMT

But still this autoexec.sas needs to be included in her program which also be included in mine.no?

Re: SAS Datasets size

Tal — Tue, 18 Jul 2017 00:30:49 GMT

We will run the du command. Thank you all again

Re: SAS Datasets size

ChrisBrooks — Tue, 18 Jul 2017 00:33:42 GMT

No it's a seperate file - you can find out more details about it at this link

Re: SAS Datasets size

Reeza — Tue, 18 Jul 2017 02:24:15 GMT

She could also have set a preference or option differently in the same session or previously. Did PROC CONTENTS show now difference between the datasets? You could also run PROC COMPARE to see what it says the differences are between the two datasets to determine any differences. The OBS option is another that can cause issues.

I'm assuming you've already checked the obvious that you both have the same versions and that the datasets created have the same number of observations and variables.

A common example is someone using EG and someone else using Base. EG defaults with VALIDVARNAME=ANY while Base defaults to VALIDVARNAME=V7. Thus the exact same code in EG and Base can provide different answers.

@Tal wrote:
But still this autoexec.sas needs to be included in her program which also be included in mine.no?

NO.

Re: SAS Datasets size

Kurt_Bremser — Tue, 18 Jul 2017 07:06:09 GMT

@Tal wrote:
I go to the unix folders through a ftp. Hers shows 3.2GB mine for example 7.8GB

You should run proc contents on both datasets and compare the outputs. On a UNIX system, proc contents will also include the physical file size in the host dependent section.

Re: SAS Datasets size

Tal — Tue, 18 Jul 2017 13:01:42 GMT

so she ran proc contents and she has the compress -char option set and i have compression-no

we also found the autoexec.sas file in one of the sharedrives. Does that now means her pc sas connects to that autoexec file and mine does not?

Or if i just use systen option compress=char ?

Re: SAS Datasets size

Kurt_Bremser — Tue, 18 Jul 2017 13:09:02 GMT

I would not use the system option. Datasets that have no (or very short) strings may even increase in size.

Use the compress=yes dataset option, and look at the log what is reported. Keep the option only if a dataset is reduced significantly in size.

What constitutes "significantly" is up to you. After all, 5% of 20 GB is still 1 GB.