BookmarkSubscribeRSS Feed
Yura2301
Quartz | Level 8

Hi all,

Have general question regarding archiving SAS data sets.

So shortly I'm working with daily backuping process, main functionality finished, now I need to archivate each of the backed up(off course compressed) sas data sets.

I made small investigation and made 2 scripts on vbs, executet from sas by X comnmand.

First vbs make ZIP archive from SAS data set.

The vbs code is small for such variant(just a few lines), archivation time is quick,  but the minus is that big data sets(>100 mb) compreesed only by 50%, so target .zip archive become approximately only 2 times smaller.

Second VBS use 7Z archivator, vbs code a little bit bigger then in first case, require 7z exe file exists on machine etc., but compress much better ~10%, so result archive ~10 times smaller then initial SAS data set.

But archivation time obviosuly a few times bigger then in case of ZIP archivation.

So question is - what is the best, maybe more efficient archivator for SAS data sets files(both data set and index files).

From another word maybe someone had similar tasks before and made deep research that results in fact that archivator XXX is the best for compressing SAS data sets?

Thanks!

6 REPLIES 6
RW9
Diamond | Level 26 RW9
Diamond | Level 26

Google: http://www.tomshardware.com/reviews/winrar-winzip-7-zip-magicrar,3436-13.html

The question arises, do you just want to keep old files, or do you want to use them in a more meaningful way?  If its just keeping old files, then compression is probably ok, however bear in mind that each file takes up room so space can be an issue, and finding things/comparisons etc.  If you want a back then maybe just buy a tape backup machine and run tape backups once in a while. 

If however you want to use the data or keep audit trails etc. then you would be best served looking into databases/warehouses.  These provided audit trails where you can rebuild the data step by step (i.e. only keep the changes from baseline), so keeping only the necessary change information, far smaller than compressing the full item each time.  The data is all in one place and accessible etc.  However there are downsides, you need to admin the system, provide some kind of ETL etc.

Yura2301
Quartz | Level 8

Hi RW9,

Yep, just need keep old files, so compression looks lilke ok.

Offcourse company has daily files backup system, tape backups.

Thanks for good explanations, I also googled about 7z before, that's why actually choose it , but I just thought that all SAS data sets has more-less same structure so maybe some other archivator is better specially for .sas7bdat and .sas7bndx files.

But looks like 7z is the best anyway:)

Thanks!

Yura2301
Quartz | Level 8

Thanks for the input!

Patrick
Opal | Level 21

If you're using a X command then you could also call 7z directly from there. No need to call a VBS which then calls 7z.

Yura2301
Quartz | Level 8

Hi Patrik,

Yes, but archiving process can fail for a some reasons, so just run 7z.exe with needed parameters probably a little bit unsecure.

I used smth like:

Set objectShell   = CreateObject("WScript.Shell")

retVal = objectShell.Run(...)

If retVal <> 0 Then

' write to log worning that archivation failed

     Else

...

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 6 replies
  • 2616 views
  • 3 likes
  • 4 in conversation