Data Integration Studio: problematic code generation

FK1 — Wed, 22 Feb 2023 11:11:09 GMT

Hello Everyone,

let me briefly give you some background infos, before explaining my issue:

SAS offers the "VARINITCHK" option, which "specifies whether a DATA step stops or continues executing when a variable is not initialized and the type of message to write to the SAS log".

Now, I am using this option in its strictest version, id est: a DATA step stops executing and writes an error message to the SAS log when a variable is not initialized.

Now my issue:

My chosen strictness level of the aforementiond option causes the cancelation of batch scheduled, (automcatically) written Data Integration Studio Jobs, since it turns out, that in some occasions the autogenerated code is not "clean". Sidenote: at least I can uphold my quality standard, since the "quality guarantee" I wanted by having only "clean" code to be run is fulfilled 🙂

BUT: how can I correct the "pollution process" of DIS? Meaning: is there a possibility to modify the etls-macros, which are being used during the code generation process of DIS?

Where exactly are the etls-Macros located in the filesystem? (I guess the name of the macros is derived from the good old days, when DIS was called "ETL Studio"). As far as I know they are "hidden" in some ominous Java classes.

For example, using the SCD2 Table loader transformation, is problematic, since it uses the "ETLS_SCDLOADER" macro to autogenerate the job code which is "polluted" since sometimes the line

call missing(of _all_);

is missing when declaring variable attributes within a data step.

Here the ERROR excerpt of the log attached to this post:

MPRINT(ETLS_SCDLOADER):   data etlstmp.W9HE3C1G;
MPRINT(ETLS_SCDLOADER):   attrib Make length = $13;
MPRINT(ETLS_SCDLOADER):   attrib Model length = $40;
MPRINT(ETLS_SCDLOADER):   attrib Type length = $8;
MPRINT(ETLS_SCDLOADER):   attrib Origin length = $6;
MPRINT(ETLS_SCDLOADER):   attrib DriveTrain length = $5;
MPRINT(ETLS_SCDLOADER):   attrib MSRP length = 8 format = DOLLAR8.;
MPRINT(ETLS_SCDLOADER):   attrib Invoice length = 8 format = DOLLAR8.;
MPRINT(ETLS_SCDLOADER):   attrib EngineSize length = 8 label = 'Engine Size (L)';
MPRINT(ETLS_SCDLOADER):   attrib Cylinders length = 8;
MPRINT(ETLS_SCDLOADER):   attrib Horsepower length = 8;
MPRINT(ETLS_SCDLOADER):   attrib MPG_City length = 8 label = 'MPG (City)';
MPRINT(ETLS_SCDLOADER):   attrib MPG_Highway length = 8 label = 'MPG (Highway)';
MPRINT(ETLS_SCDLOADER):   attrib Weight length = 8 label = 'Weight (LBS)';
MPRINT(ETLS_SCDLOADER):   attrib Wheelbase length = 8 label = 'Wheelbase (IN)';
MPRINT(ETLS_SCDLOADER):   attrib Length length = 8 label = 'Length (IN)';
MPRINT(ETLS_SCDLOADER):   attrib SCD2_CURRENT_FLAG length = 8;
MPRINT(ETLS_SCDLOADER):   attrib SCD2_VALID_FROM length = 8 format = Datetime20.;
MPRINT(ETLS_SCDLOADER):   attrib SCD2_VALID_TO length = 8 format = Datetime20.;
MPRINT(ETLS_SCDLOADER):   attrib SCD2_SURROGATE_KEY length = 8;
MPRINT(ETLS_SCDLOADER):   stop;
MPRINT(ETLS_SCDLOADER):   run;

ERROR: Variable Make is uninitialized.
ERROR: Variable Model is uninitialized.
ERROR: Variable Type is uninitialized.
ERROR: Variable Origin is uninitialized.
ERROR: Variable DriveTrain is uninitialized.
ERROR: Variable MSRP is uninitialized.
ERROR: Variable Invoice is uninitialized.
ERROR: Variable EngineSize is uninitialized.
ERROR: Variable Cylinders is uninitialized.
ERROR: Variable Horsepower is uninitialized.
ERROR: Variable MPG_City is uninitialized.
ERROR: Variable MPG_Highway is uninitialized.
ERROR: Variable Weight is uninitialized.
ERROR: Variable Wheelbase is uninitialized.
ERROR: Variable Length is uninitialized.
ERROR: Variable SCD2_CURRENT_FLAG is uninitialized.
ERROR: Variable SCD2_VALID_FROM is uninitialized.
ERROR: Variable SCD2_VALID_TO is uninitialized.
ERROR: Variable SCD2_SURROGATE_KEY is uninitialized.
NOTE: SAS variable labels, formats, and lengths are not written to DBMS tables.
NOTE: The SAS System stopped processing this step because of errors.
WARNING: The data set ETLSTMP.W9HE3C1G may be incomplete.  When this step was stopped there were 0 observations and 19 variables.
ERROR: ROLLBACK issued due to errors for data set ETLSTMP.W9HE3C1G.DATA.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.01 seconds

Re: Data Integration Studio: problematic code generation

Patrick — Wed, 22 Feb 2023 11:38:15 GMT

Short answer: You can't change it.

SAS DI Studio is written in Java. It generates SAS code metadata driven. The code snippets used to generate the code are part of the product.

topic Data Integration Studio: problematic code generation in SAS Programming

Data Integration Studio: problematic code generation

Re: Data Integration Studio: problematic code generation