Error Handling for X or SYSEXEC Shell Commands

[cghost]

I have a number of SAS programs that run over night and start with a series of x or %sysexec Windows shell commands that copy binary data files off of a server, invoke a program at the command line to export them into plain text, manipulate the plain text data files into fully-formatted SAS datasets, then grab related pdfs for each dataset and move them to a new location, using SAS to create a running table of where the PDFs land and which PDFs are related to which output SAS datasets. Imagine 3 or 4 nightly commands that loop simple commands like the following anywhere from 6 to 30 times each:

proc printto log = "C:\Users\Cghost\Logs";

run;



%let server = \\network-share\;

%let datafiles = datasetname;


%sysexec xcopy /E "&server.&datafiles."  "&localcopy.&datafiles." /Y;

x """C:\Program Files (x86)\DataPackage\Bin\Conversion.exe"" ""&localcopy.&datafiles.\ExportData_&datafiles..conv""";

I log all this using proc printto, but obviously I don't see errors or outcomes (i.e., "File copied successfully") from the command shell, so this is leaving me unaware when a file is not copied correctly for some reason or when there's an error invoking the program used to convert the proprietary binary data files to plain text. So my question is, how can I grab this information back into SAS, and more specifically, what's the best way to do so? Is there a way to pull the result text from the command shell directly back into my SAS log so that I can see the results there? Or will I need to store it as a separate file (and if so, how can I append to the same daily log file for each time I invoke x or sysexec)?

[Patrick]

@cghost

Not 100% sure but doesn't the &sysrc macro variable contain the return code of your child process?

My preference would be to use a scheduler for such tasks. If I would have to code for it with SAS only then I'd use SYSTASK as imho this command gives you the most control - and there is a STATUS variable for what you need plus combined with WAITFOR also the option to run tasks in parallel or sequentially. 

[DaveHorne]

As an alternative for xcopy, I've been using the Windows robocopy command to copy files via a unnamed pipe.  Robocopy has a long set of command line options and using the pipe provides a lot of information back to the SAS log.  Here's an example:

%let srcdir=\\dave\data\tables;
%let destir=C:\tables\;

filename robocopy pipe "robocopy &srcdir &destir /NP /R:1 /W:1" console=min;

data _null_;
   infile robocopy;
   input;
   put _infile_;
run;

filename robocopy clear;

and the SAS log:

NOTE: The infile ROBOCOPY is:
      Unnamed Pipe Access Device,
      PROCESS=robocopy \\dave\data\tables\ C:\tables\ /NP /R:1 /W:1,
      RECFM=V,LRECL=32767


-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows                              
-------------------------------------------------------------------------------

  Started : Thursday, June 7, 2018 1:47:23 AM
   Source : \\dave\data\tables\
     Dest : C:\tables\

    Files : *.*
	    
  Options : *.* /DCOPY:DA /COPY:DAT /NP /R:1 /W:1 

------------------------------------------------------------------------------

	                  39	\\dave\tables\
	  *EXTRA File 		1019.0 m	invcurr_sc.sas7bdat
	  *EXTRA File 		  44.0 m	invcurr_sc.sas7bndx
	  *EXTRA File 		 799.2 m	invcurr_sc2.sas7bdat
	  *EXTRA File 		  46.9 m	invcurr_sc2.sas7bndx
	    Newer     		  10.9 m	allps.sas7bdat
	    Newer     		   1.1 m	allps.sas7bndx
	    Newer     		  262144	atlas.sas7bdat
	    Newer     		   65536	atlas.sas7bndx
	    Newer     		 183.5 m	commente.sas7bdat
	    Newer     		  69.3 m	commente.sas7bndx
	    Newer     		  46.4 m	emailid.sas7bdat
	    Newer     		   2.1 m	fmadj_ptg.sas7bdat
	    Newer     		   65536	fmadj_ptg.sas7bndx
	    Newer     		  196608	galloc.sas7bdat
	    Newer     		  196608	goodgs.sas7bdat
	    Newer     		   97792	goodgs.sas7bndx
	    Newer     		  27.8 m	icodes.sas7bdat
	    Newer     		  196608	intl.sas7bdat
	    Newer     		   97792	intl.sas7bndx
	    Newer     		 104.8 m	invcons.sas7bdat
	    Newer     		  25.2 m	invcons.sas7bndx
	    Newer     		   1.8 g	invcurr.sas7bdat
	    Newer     		 551.8 m	invcurr.sas7bndx
	    Newer     		  18.7 m	invhard.sas7bdat
	    Newer     		   2.0 m	invhard.sas7bndx
	    Newer     		  22.0 m	invlookup.sas7bdat
	    Newer     		   5.8 m	invlookup.sas7bndx
	    Newer     		  13.3 m	invsalee.sas7bdat
	    Newer     		   3.5 m	invsalee.sas7bndx
	    Newer     		  262144	regdr.sas7bdat
	    Newer     		   97792	regdr.sas7bndx

------------------------------------------------------------------------------

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         1         0         0         0         0         0
   Files :        39        27        12         0         0         4
   Bytes :   3.741 g   2.959 g  800.80 m         0         0   1.864 g
   Times :   0:01:38   0:01:37                       0:00:00   0:00:00


   Speed :            32434077 Bytes/sec.
   Speed :            1855.892 MegaBytes/min.
   Ended : Thursday, June 7, 2018 1:49:02 AM
NOTE: 61 records were read from the infile ROBOCOPY.
      The minimum record length was 0.
      The maximum record length was 81.
NOTE: DATA statement used (Total process time):
      real time           1:38.41
      user cpu time       0.00 seconds
      system cpu time     0.00 seconds
      memory              295.71k
      OS Memory           17240.00k
      Timestamp           06/07/2018 01:49:02 AM
      Step Count                        42  Switch Count  0
      

271        
272        filename robocopy clear;
NOTE: Fileref ROBOCOPY has been deassigned.