Bulk Download all my files from SAS Ondemand

[LauraRK]

I am very embarrased but I realized what happened, I have two accounts, one I use to pretend to be a student, and one is my actual teacher account.  I was accidentally logged into the wrong account.  Nevertheless, it is somewhat of a fortunate error as it allows me to now have a more robust solution and to understand a bit better the file system for SAS Ondemand.   It also helps me to be more empathetic, and be reminded that it is easy to make small mistakes when one is thinking about harder issues and does not take a few moments to be mindful of what is going on.   I also put a bunch of time into learning Github and integrating it with R-studio cloud, so I think my next goal might be figuring that out with SAS Ondemand, https://blogs.sas.com/content/sasdummy/2019/01/17/git-in-sas/ .  @RandyMullis do you know if that is possible with SAS Ondemand?  

Meanwhile,  I think the tool @Kurt_Bremser has given us is very nice and I will definitely use it to zip and download my SAS programs in nice neat directories.

[Kurt_Bremser]

I have made it a habit to always use $HOME for the home directory. That makes code portable.

And not only that. Anything you can get from environment or automatic macro variables, or other means (e.g. functions) should be defined that way.

[LauraRK]

So use $HOME instead of ~ . Just to make sure I understand. Thanks.

[Kurt_Bremser]

Both will work, but I think the use of $HOME predates the tilde. Since I have almost 4 decades of working with UNIX, call me old school.

[Tom]

$HOME is just a reference to an environment variable.  ~ is a feature supported I believe by the shell, so it might work for some shells and not for others.  So over the years I have seen Unix systems where one or the other of these do not work.

Plus using SAS on demand (or running SAS via a process server in any way) there might well be a different userid that is running the actual SAS code than the one usered to find your home directory.  In that case it will be easier for SAS to have set the HOME environment variable than for the ~ to get interpreted properly.

[LauraRK]

Thank you all. I am just trying to make sure I understand, https://support.sas.com/ondemand/coursedataorganization.html talks about the paths. I think with the SAS OnDemand the ~ is going to work best or the /home/username/ but given I teach, the ~ is easiest because then what I do matches the students work. I have found it interesting getting used to working on a system where I do not have direct control.

[Tom]

So you want the companion UNZIP macro to go with the ZIP method in the selected solution?

You could probably remove the references to %PARMV() macro or get it from github.

%macro unzip
/*----------------------------------------------------------------------------
Extract files from a ZIP file using only SAS code
----------------------------------------------------------------------------*/
(zipfile    /* Source file */
,outdir     /* Target directory */
,dsname=unzipfiles /* Name of dataset created with filenames */
,createdir=YES  /* Use DLCREATEDIR option to make subdirectories */
);
%local macro parmerr dlcreatedir;
%let macro=&sysmacroname;
%let dlcreatedir=%sysfunc(getoption(dlcreatedir));
%parmv(zipfile,_req=1,_case=n,_words=1)
%parmv(outdir,_req=1,_case=n,_words=1)
%parmv(dsname,_case=n)
%parmv(createdir,_val=0 1)
%if (&parmerr) %then %goto quit;

*-----------------------------------------------------------------------------;
* Get list of members in the ZIP file ;
*-----------------------------------------------------------------------------;
data &dsname ;
  length fileref $8 fileno 8 memname zipfile $256.;
  zipfile = symget('zipfile');
  rc=filename(fileref,zipfile,'zip');
  if rc then call execute('%nrstr(%parmv)(zipfile,_msg=Unable to open file)');
  else do;
    did=dopen(fileref);
    if not did then call execute
       ('%nrstr(%parmv)(zipfile,_msg=Unable to open file as a zip file)')
    ;
    else do;
      do fileno=1 to dnum(did);
        memname=dread(did,fileno);
        output;
      end;
      did=dclose(did);
      rc=filename(fileref);
    end;
  end;
  drop fileref did rc;
run;
%if (&parmerr) %then %goto quit;

*-----------------------------------------------------------------------------;
* Copy members to files in target directory ;
*-----------------------------------------------------------------------------;
%if (&createdir) %then %do;
options dlcreatedir;
%end;
%else %do;
options nodlcreatedir;
%end;
data &dsname ;
  set &dsname ;
  length libref in out $8 outfile $256.;
  retain libref ' ';
  if "&sysscp"='WIN' then sep='\';
  else sep='/';
  rc1=filename(in,zipfile,'zip'
              ,catx(' ','recfm=f lrecl=512','member=',quote(trim(memname))))
  ;
  outfile=symget('outdir');
  rc2=libname(libref,outfile);
  do level=1 to countw(memname,'/')-1;
    outfile=catx(sep,outfile,scan(memname,level,'/'));
    rc2=libname(libref,outfile);
  end;
  rc2x=libname(libref);
  outfile=catx(sep,outfile,scan(memname,level,'/'));
  rc3=filename(out,outfile,,'recfm=f lrecl=512');
  rc4=fcopy(in,out);
  length msg $ 384;
  if rc4 then do;
     msg=sysmsg();
     call symputx('parmerr','1');
     put msg;
  end;
  drop libref in out ;
run;
%if (&parmerr) %then %do;
  %parmv(_msg=Not all files created);
%end;
%quit:
options &dlcreatedir ;
%mend unzip;

[RandyMullis]

Hi, @LauraRK . I spoke with the manager for SAS OnDemand for Academics this morning. He said that while the interface on the left doesn't have the Git portions exposed, the procedures and functions related to Git are fully operational. I don't use those myself but you should be able to make use of them. I just tried using some of the examples from:

https://blogs.sas.com/content/sasdummy/2019/01/17/git-in-sas/

and everything looked good.

As for $HOME vs ~, I agree with my esteemed colleagues that $HOME is the way to go. The tilde will work fine in most situations but they aren't equivalent. If you want to learn more than you'd probably ever want to know, you can read this thread:

https://askubuntu.com/questions/1177464/difference-between-home-and

Thanks,

Randy

[waverrevenue]

This was a lifesaver; thank you for this solution and the followup conversation, which helped me configure the filepath.

[corpusling]

Thank you for the code to zip up files in SAS OnDemand. 

I used the accepted solution, which worked great with text files (.html, .sas, .tsv, etc.), but not with png files (charts, for instance). With png files, the zip is generated, downloads and opens fine, but the png files in it are damaged. 

Any ideas appreciated, thank you ahead.

Here's the code:

data filelist;
length dname filename $256 dir level 8 lastmod size 8;
format lastmod datetime20.;
input dname;
retain filename ' ' level 0 dir 1;
cards4;
/home/path...
;;;;

data filelist;
modify filelist;
rc1=filename('tmp',catx('/',dname,filename));
rc2=dopen('tmp');
dir = not not rc2;
if not dir then do;
fid=fopen('tmp','i',0,'b');
lastmod=input(finfo(fid,foptname(fid,5)),NLDATM100.);
size=input(finfo(fid,foptname(fid,6)),32.);
fid=fclose(fid);
end;
else do;
dname=catx('/',dname,filename);
filename=' ';
lastmod=input(dinfo(rc2,doptname(rc2,5)),NLDATM100.);
end;
replace;
if dir;
level=level+1;
do i=1 to dnum(rc2);
filename=dread(rc2,i);
output;
end;
rc3=dclose(rc2);
run;


filename out zip "/home/path.../png.zip ";

%macro write_single(fname);

data _null_;
infile "&fname";
file out("&fname");
input;
put _infile_;
run;

%mend;

data _null_;
set filelist;
where filename ne "" and scan(filename,-1,".") = "png";
call execute('%write_single('!!catx("/",dname,filename)!!")");
run;

[Kurt_Bremser]

My macro is for text files only (and limited to a line length of 254).

See https://communities.sas.com/t5/SAS-Programming/Copying-binary-files-out-of-a-ZIP-fileref/td-p/811409 for a method to move binary files in and out of zip archives. Use the FCOPY method depending on file type.