topic Re: How to Read Load and Process Multiple .GZ files in SAS Programming

How to Read Load and Process Multiple .GZ files

prolifious — Thu, 23 May 2019 15:01:51 GMT

I have about 30 files to read and process from separate directories

'/path/2019/0421/target.sas7bdat.gz'
'/path/2019/0420/target.sas7bdat.gz'
'/path/2019/0419/target.sas7bdat.gz'
'/path/2019/0418/target.sas7bdat.gz'
...

With uncompressed .sas7bdat files, i load them with libname and reference them in a set statement within a data step.

But, that is not working with the compressed files.

As I'm trying to start small here, I began with this, which doesn't work either ( https://blogs.sas.com/content/sasdummy/2017/10/10/reading-writing-gzip-files-sas/ 😞

filename pdb32 zip '/path/2019/0421/target.sas7bdat.gz' gzip;

data temp0 ;
infile pdb32 obs = 10;
run;

proc print data = temp0 ;
run;

The log keeps yelling at me that the gzip at the end of the filename statement is an error (i turned the slashes around too, and got the same result).

Can someone lend some guidance on how I can read and process these files in a way similar to the libname - set statement method?

Thank you very much,
Jon

Re: How to Read Load and Process Multiple .GZ files

ChrisHemedinger — Thu, 23 May 2019 15:05:52 GMT

GZIP was added in SAS 9.4 Maint 5. If you're running an earlier version, then that's the reason for the error.

If you want to try this as a proof-of-concept, you could download SAS University Edition -- then use your success (I hope) as justification to update your level of SAS.

If that's not an option, you'll have to use external tools (via FILENAME PIPE is standard) to make this work. Lots of doc/papers that show how to do that.

Re: How to Read Load and Process Multiple .GZ files

prolifious — Thu, 23 May 2019 15:08:28 GMT

we're running

SAS (r) Proprietary Software 9.4 (TS1M2)

not new enough?

Re: How to Read Load and Process Multiple .GZ files

ChrisHemedinger — Thu, 23 May 2019 15:23:23 GMT

Nope -- you need TS1M5 or higher for GZ support.

Re: How to Read Load and Process Multiple .GZ files

prolifious — Thu, 23 May 2019 16:43:07 GMT

OK, thank you...I'm having trouble finding reference for mysetup: i'm using sas on linux, i write my programs in notepad++ as a .sas file and run it under the sas command.

Any leads on which version/method of FILENAME PIPE to use?

Thanks again,

Jon

Re: How to Read Load and Process Multiple .GZ files

Tom — Thu, 23 May 2019 16:54:16 GMT

So you have a compressed SAS dataset. You need to uncompress it before SAS can read it.

You could just un-compress it in the folder where it lives (assuming you have write access);

data _null_;
  infile 'gunzip mydata.sas7bdat.gz' pipe;
  input;
  put _infile_;
run;
proc contents data='mydata.sas7bdat'; 
run;

Or you might need to un-compress to a different area where you have write access. Perhaps your SAS work folder.

%let path=%sysfunc(pathname(work));
data _null_:
  infile "mkdir -p &path/temp" pipe;
  input;
  put _infile_;
run;

data _null_;
  infile "gzip -cd mydata.sas7bdat.gz > &path/mydata.sas7bdat" pipe;
  input;
  put _infile_;
run;

libname mytemp "&path/temp";

proc contents data=mytemp.mydata;
run;