I want to write a SAS/IML module that invokes ODS statements at execution time, not compilation time

[rbettinger]

I want to write a SAS/IML module that invokes ODS statements at execution time, not compilation time. Can this be done? If so, please show me via a code snippet so that I may be able to learn from it. For example:

proc iml ;

/* capture a graphic into a specified file */

run ODS_PDF( 'ods_file_containing_SGPLOT_output' ) ;

submit ;

/* SGPLOT statements here */

endsubmit ;

run ODS_PDF() ;

quit ;

The IML module ODS_PDF will manage the creation of the file 'C:\path_to_SAS_directory_containing_IML_code_to_execute\ods_file_containing_SGPLOT_output.pdf' . When the argument list to ODS_PDF() is empty, the ODS destination will be closed.

Is there some way to mask  the ODS commands from immediate compilation?

TIA,

Ross

[Tom]

I don't understand what the question is.  Isn't the fact that you have defined the module going to prevent the module from actual execution until you run it?

Is there some aspect of this that you are having trouble with?

Is it determining if the input parameter to the module is empty?  I would try using the MISSING() function.

Is it converting the input parameter to the module into part of the filename used in the ODS statement?

Is it determining what directory to tell ODS to write the file into?  How is PROC IML supposed to know what "program file" it is running?

Something else?

[rbettinger]

Tom,

Thank you for your interest in my question. I want to invoke ODS statements at execution time, not at compile time. I want to create a PDF file in the current working directory automatically, instead of having to type 'ODS PDF FILE='"..." ' for each set of graphics call. Here is an example of what I am trying to do:

libname FUZZYLIB 'C:\Users\RossMaster\Documents\My SAS Files\Fuzzy_Logic_Toolbox_SAS' ;

proc iml ;
reset storage=fuzzylib.fuzzylib ;
load module=_all_ ;

   start ods_pdf( pdf_file= ) ;
      /* purpose: create ods_pdf() to establish ODS PDF environment
       *
       * pdf_file ::= quoted character string containing name of PDF file to be created in 
       *              directory containing calling program
       *
       *              if parameter value is missing, the ODS PDF CLOSE command will be issued
       */

      if ^isEmpty( pdf_file )
      then do ;
         path      = "%sysget( SAS_EXECFILEPATH )" ;
         path_name = "%sysget( SAS_EXECFILENAME )" ;

         path_len = length( path ) - length( path_name ) ;

         file_name = cats( substr( path, 1, path_len ), pdf_file, '.pdf' ) ;

         rc = filename( 'PDF_REF', file_name ) ;

         call execute( 'ods _all_ close ;'      ) ;
         call execute( 'ods pdf file=PDF_REF ;' ) ;
      end ;
      else do ;
         call execute( 'ods pdf close ;' ) ;
         call execute( 'ods listing ;'   ) ;
      end ;

   finish ods_pdf ;

   /* test code */

   run ods_pdf( 'abcdef' ) ;

   submit ;
      proc sgplot data=sashelp.class ;
      series x=Weight y=Height / datalabel=Age datalabelpos=top markers ;
      xaxis label='Index' ;
      yaxis label='Fuzzy Entropy' ;
      run ;
   endsubmit ;

   run ods_pdf() ;

   store module=ods_pdf ;
quit ;

This code works the first time that it is used, but if the file PDF_FILE exists in the current working directory, it fails. But that is a minor fix, IMHO. I believe that I have found the solution by using CALL EXECUTE(), which masks the ODS code from execution at compile time and instead defers it to run time.

Ross

[Tom]

It doesn't "mask" the ODS code.  It GENERATES the ODS code.  There is no code to mask until it has been created, which will only happen when it runs.

[rbettinger]

Thank you for your clarification, Tom. 

[rbettinger]

Here is a SAS/IML module that I wrote to manage ODS PDF text and image capture. It is called with a character string that represents the name of the PDF file to be written in the current working directory. It must be called with a file name before the graphics are to be generated, and again after the graphics have been generated. 

libname FUZZYLIB 'C:\Users\UserID\Documents\My SAS Files\Fuzzy_Logic_Toolbox_SAS' ;

proc iml ;
reset storage=fuzzylib.fuzzylib ;
load module=_all_ ;

   start ods_pdf( pdf_file= ) ;
      /* purpose: create ods_pdf() to establish ODS PDF environment
       *
       * pdf_file ::= quoted character string containing name of PDF file to be created in 
       *              directory containing calling program
       *
       *              if parameter value is missing, the ODS PDF CLOSE command will be issued
       */

      if ^isEmpty( pdf_file )
      then do ;
         path = cats( 
                      substr( "%sysget( SAS_EXECFILEPATH )"
                            , 1
                            , length( "%sysget( SAS_EXECFILEPATH )" ) - length( "%sysget( SAS_EXECFILENAME )" )
                            )
                    , pdf_file
                    , '.pdf'     
                    ) ;

         pdf_file_exists = fileexist( path ) ; /* rc = 0 if no prior assignment */

         if pdf_file_exists then rc = fdelete( "PDF_REF" ) ;

         call execute( "filename PDF_REF " + '"' + path + '"' + ";" ) ; /* establish fresh fileref */

         call execute( 'ods _all_ close ;'      ) ;
         call execute( 'ods pdf file=PDF_REF ;' ) ;
      end ;
      else do ;
         call execute( 'ods pdf close ;' ) ;
         call execute( 'ods listing ;'   ) ;

         rc = %sysfunc( filename( PDF_REF )) ; /* release fileref */
      end ;

   finish ods_pdf ;

   /* test code */

   run ods_pdf( 'abcdefg' ) ;

   submit ;
      proc sgplot data=sashelp.class ;
      series x=Weight y=Height / datalabel=Age datalabelpos=top markers ;
      xaxis label='Weight' ;
      yaxis label='Height' ;
      run ;
   endsubmit ;

   run ods_pdf() ;

   store module=ods_pdf ;
quit ;

To repeat, the sequence of module invocations is:

run ODS_PDF( 'PDF_output_file' ) ;

/* graphics are generated here */

run ODS_PDF() ;

and a PDF file named 'PDF_output_file.pdf' will be created in the same directory in which the ODS_PDF() module was executed.

I thank KSharp and Rick Wicklin for helping me to understand how global statements are processed.

Ross

[Tom]

You are using FILEEXISTS() to test if the physical file exists and then using the FDELETE() function to delete whatever physical file the filref PDF_REF is pointing at.  There are two problems with that:

• that fileref might not have been defined yet
• that fileref might be pointing at a completely different file.

It might make more sense to run the FILENAME() function first.  You can then use the FILEREF() function to check if (A) the filename function worked and (B) the file it points to exists or not.  Then you can decide if the existing file needs to be deleted.

Example of using FILEREF().

 1          * Macro to check a fileref and report results;
 2          %macro file_chk(fileref);
 3          %put NOTE: &=fileref %sysfunc(choosec(2+
%sysfunc(sign(%sysfunc(fileref(&fileref))))
 4          ,is Defined but does NOT Exist
 5          ,is Defined and Exists
 6          ,is NOT Defined)).;
 7          %mend file_chk;
 8          
 9          * Make sure PDF_REF fileref is not defined ;
 10         %let fileref=pdf_ref;
 11         %put rc=%sysfunc(filename(fileref));
 rc=0
 12         
 13         %let path=%sysfunc(pathname(work));
 14         %let fileref=pdf_ref;
 15         * Check FILEREF ;
 16         %file_chk(&fileref)
 NOTE: FILEREF=pdf_ref is NOT Defined.
 17         * Use FILENAME() to create FILEREF ;
 18         %let rc=%sysfunc(filename(fileref,"&path/test.pdf"));
 19         %file_chk(&fileref)
 NOTE: FILEREF=pdf_ref is Defined but does NOT Exist.
 20         
 21         * Write something to the file ;
 22         data _null_;
 23           file &fileref;
 24           put 'hello';
 25         run;
 
 NOTE: The file PDF_REF is:
       (system-specific pathname), 
       (system-specific file attributes)
 
 NOTE: 1 record was written to the file (system-specific pathname).
       The minimum record length was 5.
       The maximum record length was 5.
 NOTE: DATA statement used (Total process time):
       real time           0.00 seconds
       cpu time            0.00 seconds
       
 
 26         %file_chk(&fileref)
 NOTE: FILEREF=pdf_ref is Defined and Exists.
 27         
 28         * Remove the file ;
 29         %let rc=%sysfunc(fdelete(&fileref));
 30         %file_chk(&fileref)
 NOTE: FILEREF=pdf_ref is Defined but does NOT Exist.
 31         

[Rick_SAS]

It looks like you intend this functionality as part of library that others will use.

If so, I encourage you to avoid using ODS _ALL_ CLOSE, which will cause havoc with your users' open ODS destinations.  You should use ODS PDF CLOSE only. If you want to exclude the output from other open ODS destinations, then use ODS EXCLUDE ALL prior to writing the PDF file, followed by ODS EXCLUDE NONE when you are ready to resume normal output. Or, if you know the name of the output table(s), you can use ODS EXCLUDE <nameoftable>.

For more details about why ODS CLOSE ALL is a bad programming practice in a public library, see Five reasons to use ODS EXCLUDE to suppress SAS output - The DO Loop.

[Tom]

and a PDF file named 'PDF_output_file.pdf' will be created in the same directory in which the ODS_PDF() module was executed.

Not really. 

It will create the file in whatever directory the macro variable SAS_EXECFILEPATH (if it exists) is pointing at.

[Ksharp]

I think you should post it at SAS/IML forum and calling out @Rick_SAS 

https://communities.sas.com/t5/SAS-IML-Software-and-Matrix/bd-p/sas_iml

And if you want to execute some code at execution time, check

 CALL EXECUTEFILE(filename < , encoding >);

it is more like %INCLUDE in data step.

And Rick wrote a blog about it before, Check it out:

https://blogs.sas.com/content/iml/2015/06/15/executefile.html

[Rick_SAS]

I think your confusion is that ODS statements are not IML statements. 

The ODS statements are GLOBAL statements in SAS. They are similar to TITLE, FOOTNOTE, LIBNAME, FILENAME, etc. 

Global statements are never seen by PROC IML! When SAS parses a program, global statements are sent to other parts of SAS to execute. This is true in every procedure. So, for example, in PROC REG, the TITLE and ODS statements are never seen by PROC REG. Those statements are passed to something called the "supervisor" which handles them appropriately. 

You might want to read the article "Calling a global statement inside a loop," which discusses the issue. The article discusses using a global statement inside a loop, but the concept of using a global statement inside a function is similar. IML only compiles and store IML statements. Global statements are stripped out and never seen by IML.

The article shows an example that uses CALL EXECUTE, which you are already using. 

[rbettinger]

I thank Tom, Ksharp, and Dr. Rick for their helpful insights and comments.

I have made progress in my attempt to control ODS PDF processing from within an IML module, but I'm not 100% done yet.

I adopted many of the suggestions proffered and include the revised ODS_PDF() module below:

options ls=200 ps=9999 ;

libname FUZZYLIB 'C:\Users\RossMaster\Documents\My SAS Files\Fuzzy_Logic_Toolbox_SAS' ;

proc iml ;
reset storage=fuzzylib.fuzzylib ;
/*load module=_all_ ;*/
/*remove module=( ods_pdf ) ;*/
show storage ;

   start ods_pdf( pdf_file=, verbose=0 ) ;
      /* purpose: manage ODS environment to capture PDF output into an external file
       *
       * pdf_file ::= character string containing name of PDF file to be created in 
       *              directory indicated by SAS_EXECFILEPATH macro variable 
       *
       *              if parameter value is empty, the ODS PDF CLOSE command will be issued
       *
       * verbose  ::= flag to control display of intermediate results of each test and action performed
       *
       * example:
       * proc iml ;
       *    reset storage=mylib ;
       *    load module=_all_ ;
       *
       *    run ods_pdf( 'cholesterol_histogram', 1 ) ; *** run in verbose mode ***
       *
       *    submit ;
       *       title 'Distribution of Cholesterol' ;
       *       proc sgplot data=sashelp.heart ;
       *          histogram cholesterol ;
       *       run ;
       *       title ;
       *    endsubmit ;
       *
       *    run ods_pdf() ;
       * quit ;
       */

      if ^isEmpty( pdf_file )
      then do ;
         /* create path to external file */
         ext_file = cats( 
                          substr( "%sysget( SAS_EXECFILEPATH )"
                                , 1
                                , length( "%sysget( SAS_EXECFILEPATH )" ) - length( "%sysget( SAS_EXECFILENAME )" )
                                )
                        , pdf_file
                        , '.pdf'     
                        ) ;

         ext_file = quote( ext_file ) ; /* surround external file with double quotes */

         if verbose then print ( cat( 'External PDF output file: ', ext_file )) ;

         /* use filename fcn to assign file reference to external PDF output file
         /* check precedence of file reference PDF_REF using global fcn fileref
          * and existence of ext_file.pdf using global fcn fexist
          *
          * fileref return codes:
          *    < 0 fileref is assigned but file does not exist
          *      0 fileref is assigned and file exists
          *    > 0 fileref is not assigned and file does not exist
          */
         rc = filename( 'PDF_REF', ext_file ) ;

         if verbose then print ( cat( "1. rc = filename( 'PDF_REF', ext_file ) : rc=", rc, choose( rc = 0, ' successful', ' unsuccessful' ))) ;

         fileref_flg = fileref( 'PDF_REF' ) ; /* get status of PDF_REF link to external file */

         if verbose then print ( catt( "2. fileref_flg = fileref( 'PDF_REF' ) : fileref_flg=", fileref_flg )) ;

         if fileref_flg < 0
         then do ;
            rc = filename( 'PDF_REF', '' ) ; /* clear file reference */

            if verbose then print ( catt( "3. fileref_flg > 0. rc = filename( 'PDF_REF', '' ) : rc=", rc, choose( rc = 0, ' successful', ' unsuccessful' ))) ;
         end ; 
         else
         if fileref_flg = 0
         then do ;
            /* if fileref points to existing file, delete file and release file reference*/
            if fileexist( ext_file )
            then do ;
               rc = fdelete( 'PDF_REF' ) ;

               if verbose then print ( catt( "4. fileref_flg = 0. rc = fdelete( 'PDF_REF' ) ; rc=", rc, choose( rc = 0, ' successful', ' unsuccessful' ))) ;

               rc = filename( 'PDF_REF', '' ) ;

               if verbose then print ( catt( "5. fileref_flg = 0. rc = filename( 'PDF_REF', '' ) : rc=", rc, choose( rc = 0, ' successful', ' unsuccessful' ))) ;
            end ;
         end ; 
         else
         if fileref_flg > 0
         then do ;
            if verbose then print ( catt( "6. fileref_flg > 0. Fileref is not assigned and external file" )) ;
         end ;

         /******************************************************************************/
         /* have prepared ODS environment
         /* link file reference to external file
         /* establish ODS PDF environment
         /******************************************************************************/
         stmt = catt( "rc = filename( 'PDF_REF', ", ext_file, ") ;" ) ;

         call execute( stmt ) ; /* establish fresh fileref-PDF output file linkage */

         if verbose then print ( catt( "7. rc = filename( 'PDF_REF', 'ext_file' ): rc=", rc, choose( rc = 0, ' successful', ' unsuccessful' ))) ;

         /* link fileref PDF_REF to external file to capture ods pdf output */
         call execute( 'ods listing close           ;' ) ;
         call execute( 'ods pdf file=', ext_file, ' ;' ) ;
      end ;
      else do ;
         /* close external file linked to PDF_REF */
         call execute( 'ods pdf close ;' ) ;
         call execute( 'ods listing   ;' ) ;

         rc = filename( 'PDF_REF', '' ) ; /* clear file reference */
 
         if verbose then print ( catt( "8. rc = filename( 'PDF_REF', '' ) : rc=", rc, choose( rc = 0, ' successful', ' unsuccessful' ))) ;
      end ;

   finish ods_pdf ;

   /* store latest version of ods_pdf() */

   store module=ods_pdf ;
   show storage ;

   /******************************************************************************/
   /* test code
   /******************************************************************************/

   run ods_pdf( 'Cholesterol', 1 ) ;

   submit ;
      title 'Distribution of Cholesterol' ;
      proc sgplot data=sashelp.heart ;
         histogram cholesterol ;
      run ;
      title ;
   endsubmit ;

   run ods_pdf( , 1 ) ;

   /***************************************************************************/

   run ods_pdf( 'height' ) ;

   submit ;
      title 'Histogram of Class Height' ;
      proc sgplot data=sashelp.class ;
         histogram height ;
      run ;
      title ;
   endsubmit ;
   run ods_pdf( , 1 ) ;
quit ;

The module compiles error-free and produces the first plot, "Distribution of Cholesterol", correctly. Then I invoke the ODS_PDF() module again and incur a system error (vide ods_pdf.txt). So maybe I have been a bad boy. I changed the ODS LISTING CLOSE to ODS EXCLUDE ALL as Dr. Rick suggests (vide ods_pdf_test_1.sas) and saw that ODS EXCLUDE ALL + ODS PDF File=<PDF output filename> suppresses the PDF output (vide ods_pdf_test_1.txt) but ODS LISTING CLOSE + ODS PDF File=<PDF output filename> works properly. And if I use ODS LISTING CLOSE on both SGPLOT calls, I see two histograms, as was to be expected.

Hence, I am still doing something wrong and my neuron is overloaded so I am asking for Community assistance. I see that if I use ODS LISTING control over ODS output, I strike out when I try to produce multiple graphics sequentially, so maybe I have not set up the appropriate ODS PDF environment. When I use ODS EXCLUDE, I turn off PDF output despite using the ODS PDF File= statement after ODS EXCLUDE ALL. I am confused.

Ross

[Tom · Accepted solution]

You should debug the issues around using ODS EXCLUDE etc independently from trying to develop your function.  Once you find a way that works you can then add it to your function.

I don't remember the exact details now, but I believe that using ODS EXCLUDE ALL has a nasty side effect of also excluding output to NEW destinations that are added after it has been set.

To work around it I had success reading in the SASHELP.VDEST view and generating individual ODS EXCLUDE statements for each open destination.

Here is an example using a DATA step to generate the ODS statements. I am sure you can figure out how to do it with IML code instead.

%let fname=~/example.xlsx ;

* This method fails ;
ods exclude all;
ods excel file="&fname";
proc print data=sashelp.class; run;
ods excel close;
ods exclude none;
proc import dbms=xlsx file="&fname" out=test replace;
run;
proc print data=test;
run;

* This method works ;
data _null_;
  set sashelp.vdest;
  call execute(catx(' ','ods',destination,'exclude all;'));
run;
ods excel file="&fname";
proc print data=sashelp.class; run;
ods excel close;
ods exclude none;
proc import dbms=xlsx file="&fname" out=test replace;
run;
proc print data=test;
run;

[rbettinger]

Rick, thank you for writing about the ODS EXCLUDE statement. I will avoid using ODS _ALL_ CLOSE in future work.

Tom, thank you so much for helping me with the subtleties of ODS EXCLUDE. I did not know about the SASHELP.VDEST dataset, which allows me to exclude all of the active ODS destinations under program control.

I have attached a final version of ODS_PDF() IML module for the Communities' perusal by interested parties. The file ods_pdf.sas manages the ODS environment before and after producing a PDF graphic, and the file ods_pdf_proof.sas contains several examples of use.

Ross