DATA Step, Macro, Functions and more

SAS Program : Size of .sas7bat files

Accepted Solution Solved
Reply
New Contributor
Posts: 2
Accepted Solution

SAS Program : Size of .sas7bat files

Hello,

 

I am new to the SAS community and I currently have a surprising problem …

 

When I run the program (.sas file) manually, the output tables (.sas7bdat) weigh :

  • tickets_maxxing_détails.sas7bdat = 15 Go
  • tickets_fid_détails.sas7bdat = 9 Go

 

When the program (.sas file) is started automatically with a batch (ControlM), the output tables (.sas7bdat) weigh :

  • tickets_maxxing_détails.sas7bdat = 44 Go
  • tickets_fid_détails.sas7bdat = 21 Go

I ran the tests twice and I get the same results

How is it possible ? Would there be a compression option to enable on the server ?

 

I need help from experts because I am a bit lost …

 

Marie-Sophie


Accepted Solutions
Solution
‎01-24-2018 05:39 AM
Super User
Posts: 9,550

Re: SAS Program : Size of .sas7bat files

First of all, I'm not a friend of using the global compress option, as datasets consisting solely or mainly of numeric data do not profit from compression; instead, the physical file size is often increased.

So I advise you look out for the location where the compress option is set for your interactive process and remove it. Instead use the individual option on datasets when they are created, then inspect the log if compressing makes sense (considerable reduction of phyiscal file size), and remove the compress=yes option if the effect is negligible or counter-productive.

Code written like that will then also compress in batch mode where compression is desired.

 

Also see Maxim 31, especially the last sentence.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
How to convert datasets to data steps
How to post code

View solution in original post


All Replies
Super Contributor
Posts: 498

Re: SAS Program : Size of .sas7bat files

Difficult to help without knowing what happens in the program. The difference could be caused by different config-files. Maybe the compress-option is active in batch, but not in scheduled mode.

New Contributor
Posts: 2

Re: SAS Program : Size of .sas7bat files

Posted in reply to andreas_lds

I just read the logs of the two program exécutions.

The .sas7bat files are compressed when the program is started manually.

 

here is the log of manual execution and the log of the batch execution.

Data set is compressed when the program is started manually.

Is there an option to compress the data set when the program is started with a batch ?

 

Thanks for your help,

 

log_manual.pnglog_batch.jpg

 

 

Solution
‎01-24-2018 05:39 AM
Super User
Posts: 9,550

Re: SAS Program : Size of .sas7bat files

First of all, I'm not a friend of using the global compress option, as datasets consisting solely or mainly of numeric data do not profit from compression; instead, the physical file size is often increased.

So I advise you look out for the location where the compress option is set for your interactive process and remove it. Instead use the individual option on datasets when they are created, then inspect the log if compressing makes sense (considerable reduction of phyiscal file size), and remove the compress=yes option if the effect is negligible or counter-productive.

Code written like that will then also compress in batch mode where compression is desired.

 

Also see Maxim 31, especially the last sentence.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
How to convert datasets to data steps
How to post code
Super User
Posts: 9,550

Re: SAS Program : Size of .sas7bat files

Files are stored as changing electrons, so their weight should be negligible Smiley Wink

 

Maxim 2: Read the log. Comparing the log step-by-step will give you a clue. If not, post the logs here; if they're too big to insert via {i}, attach them to the post as .txt files.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
How to convert datasets to data steps
How to post code
☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 4 replies
  • 99 views
  • 0 likes
  • 3 in conversation