Zipping binary files in SAS OnDemand

[tonybesas]

I'm using SAS OnDemand and would like to zip up lots of png files (graphs) generated through SAS PROC SGPLOT.

I tried the suggestion below:

https://communities.sas.com/t5/SAS-Software-for-Learning/Bulk-Download-all-my-files-from-SAS-Ondeman...

This worked well with text files (.html, .sas, .tsv, etc.), but not with the png files (charts, for instance). With the png files, the zip is generated, downloads and opens fine, but the png files in it are damaged. 

Any ideas appreciated, thank you ahead.

Here's the code:

data filelist;
length dname filename $256 dir level 8 lastmod size 8;
format lastmod datetime20.;
input dname;
retain filename ' ' level 0 dir 1;
cards4;
/home/path...
;;;;
data filelist;
modify filelist;
rc1=filename('tmp',catx('/',dname,filename));
rc2=dopen('tmp');
dir = not not rc2;
if not dir then do;
fid=fopen('tmp','i',0,'b');
lastmod=input(finfo(fid,foptname(fid,5)),NLDATM100.);
size=input(finfo(fid,foptname(fid,6)),32.);
fid=fclose(fid);
end;
else do;
dname=catx('/',dname,filename);
filename=' ';
lastmod=input(dinfo(rc2,doptname(rc2,5)),NLDATM100.);
end;
replace;
if dir;
level=level+1;
do i=1 to dnum(rc2);
filename=dread(rc2,i);
output;
end;
rc3=dclose(rc2);
run;
filename out zip "/home/path.../png.zip ";
%macro write_single(fname);
data _null_;
infile "&fname";
file out("&fname");
input;
put _infile_;
run;
%mend;
data _null_;
set filelist;
where filename ne "" and scan(filename,-1,".") = "png";
call execute('%write_single('!!catx("/",dname,filename)!!")");
run;
 

[Tom]

Did you try telling SAS to treat the files as BINARY by modifying these two lines in the macro?

infile "&fname";
file out("&fname");

[tonybesas]

Thank you for your reply . I just tried:

data _null_;
infile "&fname" recfm=N  ;
file out("&fname") recfm=N ;
input;
put _infile_;
run;

but got the following error:

ERROR: The '_INFILE_' INPUT/PUT statement option is inconsistent with binary mode I/O.  The execution of the DATA STEP is being terminated.

[Tom · Accepted solution]

Either used fixed length:

infile "&fname" recfm=F lrecl=256  ;
file out("&fname") recfm=F lrecl=256;

Or perhaps try using FCOPY() function instead.

[tonybesas]

Thank you! This worked very well! Here's the code:

data filelist;
  length dname filename $256 dir level 8 lastmod size 8;
  format lastmod datetime20.;
  input dname;
  retain filename ' ' level 0 dir 1;
cards4;
/home/<your username in SAS OnDemand>/<your folder>
;;;;

data filelist;
  modify filelist;
  rc1=filename('tmp',catx('/',dname,filename));
  rc2=dopen('tmp');
  dir = not not rc2;
  if not dir then do;
    fid=fopen('tmp','i',0,'b');
    lastmod=input(finfo(fid,foptname(fid,5)),NLDATM100.);
    size=input(finfo(fid,foptname(fid,6)),32.);
    fid=fclose(fid);
  end;
  else do;
    dname=catx('/',dname,filename);
    filename=' ';
    lastmod=input(dinfo(rc2,doptname(rc2,5)),NLDATM100.);
  end;
  replace;
  if dir;
  level=level+1;
  do i=1 to dnum(rc2);
    filename=dread(rc2,i);
    output;
  end;
  rc3=dclose(rc2);
run;

filename out zip "/home/<your username in SAS OnDemand>/<your folder>/png.zip";

%macro write_single(fname);

data _null_;
infile "&fname" recfm=F lrecl=256  ;  /* this is specific to BINARY files */
file out("&fname") recfm=F lrecl=256; /* this is specific to BINARY files */
input;
put _infile_;
run;

%mend;

data _null_;
set filelist;
where filename ne "" and scan(filename,-1,".") = "png";
call execute('%write_single('!!catx("/",dname,filename)!!")");
run;

[Reeza]

How are you generating the output? Worth wrapping it all in ODS PACKAGE to have it zipped for you? Just tested that and it works fine for the images with ODS PACKAGE.

https://support.sas.com/resources/papers/proceedings09/018-2009.pdf

[tonybesas]

Thank you for this, Reeza, I'll read that document and try out the solution!

[Reeza]

I kind of like that option as it's simple to add in to a report process so that you get a zip file for the full report. Makes it easy if you're automating a report that runs monthly for example.

[yabwon]

Try this:

/* list all files in your directory */

data filelist;
run;
data filelist;
  length root dname $ 2048 filename $ 256 dir level 8;
  input root;
  retain filename dname ' ' level 0 dir 1;
cards4;
</your/directory/with files>
;;;;
run;

data filelist;
  modify filelist;
  rc1=filename('tmp',catx('/',root,dname,filename));
  rc2=dopen('tmp');
  dir = 1 & rc2;
  if dir then 
    do;
      dname=catx('/',dname,filename);
      filename=' ';
    end;
  replace;

  if dir;

  level=level+1;

  do i=1 to dnum(rc2);
    filename=dread(rc2,i);
    output;
  end;
  rc3=dclose(rc2);
run;

proc sort data=filelist;
  by root dname filename;
run;

/* print out files list too see if you have all you want */
proc print data=filelist;
run;



/* name the zip file you want to zip into, e.g. */
%let addcntzip = /home/myuser/myZIPfile.zip;

data _null_;

  set filelist; /* loop over all files */
  if dir=0;

  rc1=filename("in" , catx('/',root,dname,filename), "disk", "lrecl=1 recfm=n");
  rc1txt=sysmsg();
  rc2=filename("out", "&addcntzip.", "ZIP", "lrecl=1 recfm=n member='" !! catx('/',dname,filename) !! "'");
  rc2txt=sysmsg();

  do _N_ = 1 to 6; /* push into the zip...*/
    rc3=fcopy("in","out");
    rc3txt=sysmsg();
    if fexist("out") then leave; /* if success leave the loop */
    else sleeprc=sleep(0.5,1); /* if fail wait half a second and retry (up to 6 times) */
  end;

  rc4=fexist("out");
  rc4txt=sysmsg();

/* just to see errors */
  put _N_ @12 (rc:) (=);

run;

Bart

[tonybesas]

This worked great, thank so much! It zipped up everything in my folder, including the text files and the binary files as well.