How to automatically syntax check generated SAS code?

[Patrick]

Hi all,

I've got a SAS program that generates .sas files.

I'd like to add some logic that syntax checks these .sas files without actually executing them (=just the compilation portion). 

There is NOEXEC for Proc SQL but I can't find a more generic solution. 

Background

I'm dealing with a SAS solution that comes with a lot of config tables for site specific configs. We're using Git for version control - but that's not great for binary objects like SAS files.

The chosen approach is to generate .sas files from these SAS config tables and version control/deploy these scripts.

I'm using an extended version of @SASJedi's code found here:  https://blogs.sas.com/content/sastraining/2016/03/11/jedi-sas-tricks-data-to-data-step-macro/ 

I provide multiple option to developers for code generation and I'm also providing macro parameters that allow to syntax check the generated code. 

I'm struggling to find a way to syntax check code for the use case that generates code as below:

%cust_solution_env_init();
libname outlib "&root_solution/irm/input_area/base/global";
data outlib.CONFIGURATION_SET;
  infile datalines dsd truncover;
  input CONFIG_SET_ID:$32. VALID_FROM_DTTM:NLDATM. VALID_TO_DTTM:NLDATM. CONFIGURATION_SET_DESC:$64.;
  format VALID_FROM_DTTM NLDATM. VALID_TO_DTTM NLDATM.;
  label CONFIG_SET_ID="Config Set Id" VALID_FROM_DTTM="Valid From Datetime" VALID_TO_DTTM="Valid To Datetime" CONFIGURATION_SET_DESC="Configuration Set Description";
datalines4;
AAA,01 January 2004 00:00:00,31 December 9999 23:59:59,aaa
BBB,01 January 2004 00:00:00,31 December 9999 23:59:59,bbb
ccc,01 January 2004 00:00:00,31 December 9999 23:59:59,cccc
ddd,01 January 2004 00:00:00,31 December 9999 23:59:59,ddd
;;;;
proc datasets lib=outlib nolist nowarn;
  modify CONFIGURATION_SET;
  ic create PRIM_KEY = Primary Key (CONFIG_SET_ID VALID_FROM_DTTM);
  ic create Not Null (VALID_TO_DTTM);
  Index create PRIM_KEY=( CONFIG_SET_ID VALID_FROM_DTTM ) / Unique Updatecentiles=5;
run;quit;
libname outlib clear;

Question

Can you think of a way how I could %include above code without executing it (so the data step doesn't execute and creates/overwrites the table)?

Thanks,

Patrick

[PaigeMiller]

You could run the generated code through ChatGPT!!!

😆😅🤣😂

That was a joke ... I don't really have a good answer ...

[Patrick]

@PaigeMiller 

🤣
I'm really impressed - and at the same time concerned - by ChatGPT.

I believe we'll remember the advent of ChatGPT the same as the more senior ones of us remember the time Google first appeared and the difference it made.

Btw: If you copy/paste below into ChatGPT then you'll get a wrong answer:
I've got a SAS program that generates .sas files.
I'd like to add some logic that syntax checks these .sas files without actually executing them (=just the compilation portion).
There is NOEXEC for Proc SQL but I can't find a more generic solution.

[Ksharp]

Add this option before your code.

options obs=0;

[Astounding]

There are a few unseen pieces here, and I definitely mixed up the purpose of some of the sections.  But this approach might work.

Include the code in the macro definition, and change it slightly:

%macro cust_solution_env_init(root_solution=&root_solution);

Then instead of limiting the output, change its destination.  Developers could call the code using:

%cust_solution_env_init (root_solution=work)

[Quentin · Accepted solution]

For syntax checking, you can probably use options obs=0 noreplace.

But that will literally just check syntax.  It won't detect problems in your data, such as if the data violates any of the integrity constraints.  So I like astounding's idea of putting this in a macro, and giving users the option to run it pointed toward WORK as the output directory.  That way a test run tests not only the syntax, but also the data. 

[Patrick]

Accepted as solution as it answers the question I've asked.

Appreciated also all the other contributions with additional suggestions/things to think about.

[yabwon]

Maybe something like this (not a silver bullet but an option): 

I had to concoct some data before anything else:

/* This is just a shell macro */
%macro cust_solution_env_init();
      %put A shell macro!;
%mend;


/* A file containing your code (I replaced datalines4 with datalines in your code to read it in with cards4, you already have it in a file.) */
filename f TEMP;
data _null_;
infile cards4;
file f;
input;
put _infile_;
cards4;
%cust_solution_env_init();
libname outlib "&root_solution/";
data outlib.CONFIGURATION_SET;

 /*if then do; end; */   /* <- un-comment ERROR line */

  infile datalines dsd truncover;
  input CONFIG_SET_ID:$32. VALID_FROM_DTTM:NLDATM. VALID_TO_DTTM:NLDATM. CONFIGURATION_SET_DESC:$64.;
  format VALID_FROM_DTTM NLDATM. VALID_TO_DTTM NLDATM.;
  label CONFIG_SET_ID="Config Set Id" VALID_FROM_DTTM="Valid From Datetime" VALID_TO_DTTM="Valid To Datetime" CONFIGURATION_SET_DESC="Configuration Set Description";
datalines;
AAA,01 January 2004 00:00:00,31 December 9999 23:59:59,aaa
BBB,01 January 2004 00:00:00,31 December 9999 23:59:59,bbb
ccc,01 January 2004 00:00:00,31 December 9999 23:59:59,cccc
ddd,01 January 2004 00:00:00,31 December 9999 23:59:59,ddd
eee,01 January 2004 00:00:00,broken data line,ddd
;
proc datasets lib=outlib nolist nowarn;
  modify CONFIGURATION_SET;
  ic create PRIM_KEY = Primary Key (CONFIG_SET_ID VALID_FROM_DTTM);
  ic create Not Null (VALID_TO_DTTM);
  Index create PRIM_KEY=( CONFIG_SET_ID VALID_FROM_DTTM ) / Unique Updatecentiles=5;
run;quit;
libname outlib clear;
;;;;
run;

Ok, so we assume  that file F contains sas code (with datalines), you could (as @Astounding suggested) set a temporary directory for data (to prevent overwriting existing data):

options dlcreatedir;
libname _ "%sysfunc(pathname(work))/test_root_slution";
%let root_solution = %sysfunc(pathname(work))/test_root_slution;
options nodlcreatedir;

and then in your current session (with all macrovariables and macros required) run something like:

data _null_;
rc = doSubL('options notes source NOReplace; /* noReplace just in case some library re-assignments went wrong */ 
%include f / source2;');
put rc=;
run;

The DoSubL() will run it in "side session" so in the main one no libraries, macros or filenames wont be affected, and you will see in the log if the code went "clear" or with errors (uncommend the error line from test code file). 

[EDIT:] I added NOReplace option.

Bart

[SASJedi]

@Patrick Have a look at the DMSSYNCHK SAS system option. 

This code worked well for me in PC SAS (SAS 9.4M8) and on the Compute Server in SAS Viya 2023.01:

options DMSSYNCHK;
put; /* Deliberately introduces an error */
%include "~/testme.sas";

[Patrick]

@Quentin, @Ksharp 

obs=0 would still run code that replaces the table so not suitable for my use case.

@SASJedi 

Using options DMSSYNCHK; still created the table when I run a test case out of SAS EG.

@Astounding , @yabwon 

Pointing the libref to WORK for testing is pretty much where I ended up with.
I've implemented multiple options. Here one where this gets directly added to the generated script. 
For my use case I don't need to run the script for replacing a table in the environment where I generate the script (because the script would always generate the exact table that got used to generate it).
Adding below syntax will generate a libname pointing to work for the environment where the script got generated. In below code "myhost" would of course be the name of the actual compute server.

%if &SYSHOSTNAME = myhost %then
  %do;
    libname outlib "%sysfunc(pathname(work))";
  %end;

data outlib.test;
  infile datalines truncover;
  input test;
  datalines;
123
;

[Astounding]

One more idea to consider ... add a global macro variable &mode.  When &mode is production, the code runs as is.  When it is &syntax_check, you automatically add the code you want.  When it is &testing, you automatically add another set of code for debugging purposes.

It obviously takes more planning to put this in place, but the work doesn't have to be done all at once.  Pieces can be added over time without affecting the production code.

[Quentin]

---

@Patrick wrote:

@Quentin, @Ksharp 

obs=0 would still run code that replaces the table so not suitable for my use case.

Please note I mentioned the NOREPLACE system option as well. That option prevents SAS from replacing permanent datasets.  By setting obs=0 NOREPLACE, you're setting the same options that SAS sets automatically when it enters syntax check mode.

That said, I like the suggestion to point the library to WORK  (or perhaps better a subdir created in WORK for this purpose, to avoid collisions with work datasets, as @yabwon showed),  as that will allow your code to actually process the data, and detect any execution-time errors.

[SASJedi]

@Patrick When running in a client-server environment, use a combination of ERRORCHECK=STRICT and SYNTAXCHECK instead of DMSSYNCHK.

options ERRORCHECK=STRICT SYNTAXCHECK;
put; /* Deliberately introduces an error */
put; /* Deliberately introduces an error */
%include "~/testme.sas";