topic Re: Create zip file of all files in a folder in SAS Programming

Create zip file of all files in a folder

DarrylLawrence — Mon, 18 Nov 2019 14:29:46 GMT

Good day

I have a folder that contains multiple PDF files. I want to create a zip file that must be emailed to someone.

When I run this code, it creates the zip file but it only contains the last file written from the folder location.

I pasted the log below the code - you can see the code referencing each PDF file but it looks like the individual pdf files are overwritten as the code runs and the zip file only contains the last file.

How can I create one zip file that contains all the PDF documents in the (unix) folder?

filename source "/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf";
filename tozip ZIP "/sas/lsf/Eighty_Twenty/Inputs/TFG_EIGHTY20_OPM_&FILEDATE..pdf.gz" ;

data _null_;
infile source;
file tozip ;
input;
put _infile_ ;
run;

NOTE: The infile SOURCE is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C93 20191028.pdf,
File List=/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf,
Owner Name=darrylla,Group Name=sas,
Access Permission=-rw-rw-r--,
Last Modified=28Oct2019:13:40:22,
File Size (bytes)=167015

NOTE: The file TOZIP is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/TFG_EIGHTY20_OPM_20191020.pdf.gz

NOTE: The infile SOURCE is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C94 20191028.pdf,
File List=/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf,
Owner Name=darrylla,Group Name=sas,
Access Permission=-rw-rw-r--,
Last Modified=28Oct2019:13:40:46,
File Size (bytes)=165533

NOTE: The infile SOURCE is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C95 20191028.pdf,
File List=/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf,
Owner Name=darrylla,Group Name=sas,
2 The SAS System 13:46 Monday, November 18, 2019

Access Permission=-rw-rw-r--,
Last Modified=28Oct2019:13:41:10,
File Size (bytes)=166410

NOTE: The infile SOURCE is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C96 20191028.pdf,
File List=/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf,
Owner Name=darrylla,Group Name=sas,
Access Permission=-rw-rw-r--,
Last Modified=28Oct2019:13:41:34,
File Size (bytes)=166450

NOTE: The infile SOURCE is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C97 20191028.pdf,
File List=/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf,
Owner Name=darrylla,Group Name=sas,
Access Permission=-rw-rw-r--,
Last Modified=28Oct2019:13:41:58,
File Size (bytes)=166928

NOTE: The infile SOURCE is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C98 20191028.pdf,
File List=/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf,
Owner Name=darrylla,Group Name=sas,
Access Permission=-rw-rw-r--,
Last Modified=28Oct2019:13:42:22,
File Size (bytes)=163935

NOTE: The infile SOURCE is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Credit Model Report_C100 20191028.pdf,
File List=/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf,
Owner Name=darrylla,Group Name=sas,
Access Permission=-rw-rw-r--,
Last Modified=28Oct2019:13:39:33,
File Size (bytes)=164781

NOTE: The infile SOURCE is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Credit Model Report_C101 20191028.pdf,
File List=/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf,
Owner Name=darrylla,Group Name=sas,
Access Permission=-rw-rw-r--,
Last Modified=28Oct2019:13:39:39,
File Size (bytes)=164970

NOTE: The infile SOURCE is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Credit Model Report_C102 20191028.pdf,
File List=/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf,
Owner Name=darrylla,Group Name=sas,
Access Permission=-rw-rw-r--,
Last Modified=28Oct2019:13:39:45,
File Size (bytes)=165148

NOTE: The infile SOURCE is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Credit Model Report_C103 20191028.pdf,
File List=/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf,
Owner Name=darrylla,Group Name=sas,
Access Permission=-rw-rw-r--,
Last Modified=28Oct2019:13:39:51,
3 The SAS System 13:46 Monday, November 18, 2019

File Size (bytes)=165113

NOTE: The infile SOURCE is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Credit Model Report_C104 20191028.pdf,
File List=/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf,
Owner Name=darrylla,Group Name=sas,
Access Permission=-rw-rw-r--,
Last Modified=28Oct2019:13:39:57,
File Size (bytes)=163764

NOTE: The infile SOURCE is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Credit Model Report_C99 20191028.pdf,
File List=/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf,
Owner Name=darrylla,Group Name=sas,
Access Permission=-rw-rw-r--,
Last Modified=28Oct2019:13:39:27,
File Size (bytes)=165667

NOTE: The infile SOURCE is:
Filename=/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty_Rapid_Input_Validation_20191029.pdf,
File List=/sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf,
Owner Name=darrylla,Group Name=sas,
Access Permission=-rw-rw-r--,
Last Modified=29Oct2019:09:49:13,
File Size (bytes)=100433

NOTE: 726 records were read from the infile SOURCE.
The minimum record length was 0.
The maximum record length was 1997.
NOTE: 748 records were read from the infile SOURCE.
The minimum record length was 0.
The maximum record length was 2113.
NOTE: 761 records were read from the infile SOURCE.
The minimum record length was 0.
The maximum record length was 1997.
NOTE: 777 records were read from the infile SOURCE.
The minimum record length was 0.
The maximum record length was 1997.
NOTE: 769 records were read from the infile SOURCE.
The minimum record length was 0.
The maximum record length was 1997.
NOTE: 756 records were read from the infile SOURCE.
The minimum record length was 0.
The maximum record length was 1997.
NOTE: 734 records were read from the infile SOURCE.
The minimum record length was 0.
The maximum record length was 1806.
NOTE: 738 records were read from the infile SOURCE.
The minimum record length was 0.
The maximum record length was 2707.
NOTE: 754 records were read from the infile SOURCE.
The minimum record length was 0.
The maximum record length was 2228.
NOTE: 728 records were read from the infile SOURCE.
The minimum record length was 0.
The maximum record length was 1919.
NOTE: 712 records were read from the infile SOURCE.
The minimum record length was 0.
4 The SAS System 13:46 Monday, November 18, 2019

The maximum record length was 1861.
NOTE: 746 records were read from the infile SOURCE.
The minimum record length was 0.
The maximum record length was 1954.
NOTE: 470 records were read from the infile SOURCE.
The minimum record length was 0.
The maximum record length was 1639.
NOTE: 9419 records were written to the file TOZIP.
The minimum record length was 0.
The maximum record length was 2707.
NOTE: DATA statement used (Total process time):
real time 0.13 seconds
cpu time 0.06 seconds

Re: Create zip file of all files in a folder

Jagadishkatam — Mon, 18 Nov 2019 14:43:24 GMT

First, please create the zip folder in unix as below by using the zip command. Please go to path /sas/lsf/Eighty_Twenty/Inputs/PDF and then open the unix terminal from here and then type the below command

zip -r allpdf.zip /sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf

The above step will create a allpdf.zip file within /sas/lsf/Eighty_Twenty/Inputs/PDF

Then send this file from sas using the below code

filename pdf email to='email address' subject='zip pdf file' attach="/sas/lsf/Eighty_Twenty/Inputs/PDF/allpdf.zip" ;

data _null_;
file tozip ;
input;
put 'zip file of pdf' ;
run;

Re: Create zip file of all files in a folder

DarrylLawrence — Mon, 18 Nov 2019 14:50:01 GMT

Thanks for the quick reply

Unfortunately I cannot logon to the terminal. I do not have access.

Also, this has to be an automated process that runs every month.

I can run the unix command in SAS but it does not create the zip file.

Is there something wrong with the syntax?

data _null_;
call system("zip -r allpdf.zip /sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf");
run;

Re: Create zip file of all files in a folder

Jagadishkatam — Mon, 18 Nov 2019 15:00:35 GMT

Could you please try the below code and confirm if it is generating the zip file.

data _null_;
call system("zip -r /sas/lsf/Eighty_Twenty/Inputs/PDF/allpdf.zip /sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf");
run;

Re: Create zip file of all files in a folder

Kurt_Bremser — Mon, 18 Nov 2019 15:14:59 GMT

When you need to know what an external command really does, use the filename pipe method:

filename oscmd pipe "zip -r allpdf.zip /sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf 2>&1";

data _null_;
infile oscmd;
input;
put _infile_;
run;

All responses (regular and error output) will end up in the SAS log.

Re: Create zip file of all files in a folder

Reeza — Mon, 18 Nov 2019 17:05:05 GMT

What about using ODS PACKAGE?

https://blogs.sas.com/content/sasdummy/2014/01/28/create-zip-ods-package/

Re: Create zip file of all files in a folder

DarrylLawrence — Tue, 19 Nov 2019 06:06:18 GMT

Hi there
I tried ods package - but you can only attach one file. I need to automatically zip all the files in the folder on a monthly basis.
Thanks
Darryl

Re: Create zip file of all files in a folder

DarrylLawrence — Tue, 19 Nov 2019 06:07:18 GMT

Thanks for the tip.
I tried this yesterday but it failed. I have contacted my UNIX dba to find out of the correct syntax as it looks like the zip command does not work.
22 GOPTIONS ACCESSIBLE;
23 filename oscmd pipe "zip -r allpdf.zip /sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf 2>&1";
24
25 data _null_;
26 infile oscmd;
27 input;
28 put _infile_;
29 run;

NOTE: The infile OSCMD is:
Pipe command="zip -r allpdf.zip /sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf 2>&1"

/usr/bin/ksh: zip: not found.
NOTE: 1 record was read from the infile OSCMD.
The minimum record length was 30.
The maximum record length was 30.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

Re: Create zip file of all files in a folder

Kurt_Bremser — Tue, 19 Nov 2019 06:32:15 GMT

See how filename pipe immediately gives us a clue?

There are two possibilities here:

zip is not installed on your system
zip is installed, but can't be found in the $PATH of the SAS session

Once it is made sure that zip is installed, as an old UNIX hand I recommend to use the absolute system path to the executable in all scripts, so you would write something like

filename oscmd pipe "/usr/bin/zip -r allpdf.zip /sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf 2>&1";

(/usr/bin/zip is the pathname for zip on AIX)

That will make it work, regardless of the contents of $PATH.

You should also consider to use gzip instead of zip when working on a UNIX system.

Re: Create zip file of all files in a folder

DarrylLawrence — Tue, 19 Nov 2019 07:09:17 GMT

My UNIX DBA helped me with this code.

We must use the "tar" command instead of "zip" as zip is a windows based utility.

/************************************************************************/

/* Zip the PDF Documents */

/************************************************************************/

data _null_;

call system("tar -cvRf- /sas/lsf/Eighty_Twenty/Inputs/PDF/*.pdf |compress>>/sas/lsf/Eighty_Twenty/Inputs/Online_Models_Summary.tar.Z");

run;

Re: Create zip file of all files in a folder

Kurt_Bremser — Tue, 19 Nov 2019 07:16:12 GMT

Wow, that's old school.

Just FYI:

The tar (short for "tape archive" - yes, really) creates a sequential stream of data (with filenames interspersed), and that stream is then fed (via the pipe) into the (equally old) compress utility, which does the compression. Therefore it's canonical to use the two-stage filename extension .tar.Z

Re: Create zip file of all files in a folder

Reeza — Tue, 19 Nov 2019 07:31:13 GMT

In the post he adds both the csv and readme files.

Did you try it and it not work? Are the files originally generated by SAS? If so this could streamlines it quite nicely

@DarrylLawrence wrote:
Hi there
I tried ods package - but you can only attach one file. I need to automatically zip all the files in the folder on a monthly basis.
Thanks
Darryl

Re: Create zip file of all files in a folder

DarrylLawrence — Tue, 19 Nov 2019 07:34:16 GMT

Yep - old school.

Someone else said I must try ODS Package -

https://blogs.sas.com/content/sasdummy/2014/01/28/create-zip-ods-package/

I tried this and it works - adding more than one file at a time. I will just write a macro for the date in the filename - will always be the 20th of the month

ods package(reports) open ;

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C93 20191028.pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C94 20191028.pdf";

ods package (reports) publish archive properties

(archive_name="Reports.zip"

archive_path="/sas/lsf/Eighty_Twenty/Inputs/PDF/");

ods package (reports) close;

Re: Create zip file of all files in a folder

DarrylLawrence — Tue, 19 Nov 2019 07:34:43 GMT

I tried this and it works - adding more than one file at a time. I will just write a macro for the date in the filename - will always be the 20th of the month

ods package(reports) open ;

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C93 20191028.pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C94 20191028.pdf";

ods package (reports) publish archive properties

(archive_name="Reports.zip"

archive_path="/sas/lsf/Eighty_Twenty/Inputs/PDF/");

ods package (reports) close;

Re: Create zip file of all files in a folder

DarrylLawrence — Tue, 19 Nov 2019 07:54:35 GMT

This works.

The filenames to zip will be the same every month except for the date in the filename. I can put this in a macro variable and then reference the rundate in the ODS Package statements.

%let rundate=20191028;

ods package(reports) open ;

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C93 &rundate..pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C94 &rundate..pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C95 &rundate..pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C96 &rundate..pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C97 &rundate..pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Cash Model Report_C98 &rundate..pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Credit Model Report_C99 &rundate..pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Credit Model Report_C100 &rundate..pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Credit Model Report_C101 &rundate..pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Credit Model Report_C102 &rundate..pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Credit Model Report_C103 &rundate..pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty Credit Model Report_C104 &rundate..pdf";

ods package (reports) add file="/sas/lsf/Eighty_Twenty/Inputs/PDF/Eighty_Twenty_Rapid_Input_Validation_&rundate..pdf";

ods package (reports) publish archive properties

(archive_name="Online_Models_Summary_&rundate..zip"

archive_path="/sas/lsf/Eighty_Twenty/Inputs/PDF/");

ods package (reports) close;