topic Re: Compress option doesn't work sometimes in SAS Programming

Compress option doesn't work sometimes

Krohn — Fri, 06 Apr 2018 00:48:53 GMT

Hello,

I'm still looking for this information:

Why sometimes compression is disabled and sometimes it doesn't?

NOTE: Compression was disabled for data set WORK.ONE because compression would increase the size of the data set.
NOTE: Compressing data set WORK.ONE increased size by 100.00 percent.

THANKS !!!!!!

Re: Compress option doesn't work sometimes

Astounding — Fri, 06 Apr 2018 01:10:18 GMT

It's not clear what you are asking for.

Did you read the note? SAS told you why.

Are you asking why compression sometimes increases the size of a data set?

Are you asking what process SAS uses to determine if compression would save anything?

Are you asking if there are any other factors that SAS considers when deciding whether or not to compress?

Are you asking when you as the programmer can control whether compression takes place?

Re: Compress option doesn't work sometimes

Krohn — Fri, 06 Apr 2018 01:51:48 GMT

I want to know if there's a way to never let it increase.

Like always disabling compression when COMPRESS would increase.

Re: Compress option doesn't work sometimes

Reeza — Fri, 06 Apr 2018 02:21:22 GMT

That is what it's supposed to do. The first error message:

NOTE: Compression was disabled for data set WORK.ONE because compression would increase the size of the data set.

You don't usually see the second message so I'm confused about that. The docs have an example and it DOES NOT have that message. Maybe you have an older version of SAS?

The documentation explains how compression can make the file bigger in the last section here:

http://documentation.sas.com/?docsetId=lrcon&docsetTarget=p0y0x1j67vtqhnn1on1cs7trim16.htm&docsetVersion=9.4&locale=en#n1mv3asl82nw64n0zpd8v283k0mr

Re: Compress option doesn't work sometimes

Krohn — Fri, 06 Apr 2018 02:49:14 GMT

That's the point......

I don't know why sometimes I get 1st NOTE and sometimes 2nd. It should always disable compress, not letting it increase.

Then many data sets have negative PCOMPRESS (eg: -43%), consuming disk.

Re: Compress option doesn't work sometimes

Reeza — Fri, 06 Apr 2018 02:54:03 GMT

No, it's not always disabled, but there is a rule.

It was in the link, I pasted the relevant section below for you:

When a request is made to compress a data set, SAS attempts to determine whether compression will increase the size of the file. SAS examines the lengths of the variables. If, due to the number and lengths of the variables, it is not possible for the compressed file to be at least 12 bytes (for a 32-bit host) or 24 bytes (for a 64-bit host) per observation smaller than an uncompressed version, compression is disabled and a message is written to the SAS log.

@Krohn wrote:

That's the point......

I don't know why sometimes I get 1st NOTE and sometimes 2nd. It should always disable compress, not letting it increase.

Then many data sets have negative PCOMPRESS (eg: -43%), consuming disk.

Re: Compress option doesn't work sometimes

Tom — Fri, 06 Apr 2018 06:07:44 GMT

It will not compress when it is impossible that compression will save space.

Otherwise it actually runs the compression algorithm and then reports the results. To know how well it worked it needs to process the whole file. At that point to go back and turn OFF the compression it would have to re-process the file again and SAS does not have any method to do that.

Re: Compress option doesn't work sometimes

ballardw — Fri, 06 Apr 2018 20:59:42 GMT

@Reeza

Here is example code to generate a similar warning:

data work.two (compress=char);
   length x y $20;
   input x y;
   datalines;
ab cd
;
run;

With that example, and changing only the length of the two variables, lengths 7 and greater generate the Increased size by 100.00 percent..

Re: Compress option doesn't work sometimes

Reeza — Fri, 06 Apr 2018 21:39:40 GMT

I agree it can happen but in that case you don't get both messages, the OP's original post made it seem like that.

Given the rule I posted earlier, I suspect this would only ever occur with really small files and personally, a 28k set being double in size doesn't concern me at all.