issue with sas studio flow execution

[acordes]

I'm working on SAS Viya 4 and I'm having the following issue with a flow in sas studio.

It only runs successfully until the end if and only if I manually run the first 2 flow items (selecting the 3rd item and choosing "Run to node") prior to running the rest ("Run to node" or "Run" complete) of the flow.

In case I run the complete flow or anything ("Run to node") from the 3rd element onwards as a first action, I got the attached error message. 

This behavior prevents me from scheduling the flow successfully as a job. 

And there is not even a log available. It throws out this alert and the cas session turns crazy forcing me to restart. 

And I cannot find a workaround, I've tried without success to split both parts into swimlanes.

 

[SASKiwi]

This looks more like an issue for Tech Support. Without the code or a SAS log we have no evidence on which to offer suggestions. About the only thing I can think of would be build the flow again from scratch in case your current version has been corrupted in some way. Tech Support is likely a quicker option.

[acordes]

Hi @BeverlyBrown, I've found your article https://communities.sas.com/t5/SAS-Communities-Library/SAS-Studio-Custom-Steps-Code-snippets-for-ste... that probably points toward a solution for my problem.

But I'd need further need from your side. How can I fix it or is is a problem by design?

Thanks a lot in advance. 

[Patrick]

The error message is clear but why this only happens if you run the whole flow and it does not happen if you run in two steps (first two nodes, node 3 to end) is confusing. 

It's certainly worth raising with SAS TS. 

What also could help for debugging is to extract the code following How To Get the Code From a SAS Studio Flow and Why It Matters 

1. What happens if you run the extracted code via a SAS Studio code window?

2. If this also fails inspect the code at the end of node 2 and the beginning of node 3 for an explanation - which then in turn might also give you an idea for a fix/workaround like an extra node that assigns the library that's apparently not (no more) assigned when running all at once.

...and share the flow definition, the extracted code and your analysis with SAS Tech Support asking for an explanation. May-be there is just something corrupted in the flow definition but may-be it's also a real bug and something SAS needs to know about so they can fix it.

[acordes]

Thanks @Patrick 

Following on your link I've created a sas code from the flow and it executes successfully after starting sas studio.

BUT the only the first 'steps' are executed, the rest of the flow is not written to the sas code. 

I don't understand why. 

[acordes]

Probably it's because of the length restriction of 32167 for the code variable in the root table. 

[BrunoMueller · Accepted solution]

Find attached a SAS program that does not have the limitation of the 32k char variable.

[Tom]

You can simplify making the file by using PARMCARDS.  No need for the complicated data step.

filename req temp;
options parmcards=req ;
parmcards4;
{
  "reference": {
    "type": "content",
    "path": "&flowname",
    "mediaType": "application/vnd.sas.dataflow"
  },
  "initCode": true,
  "wrapperCode": false
}
;;;;

And if you do need to replace the macro variable references it is probably going to be easier to either just write the file using a data step with string literals.

filename req temp;
data _null_;
  file req;
  put '{"reference": {"type": "content","path": "' "&flowname" '"' 
      ',"mediaType": "application/vnd.sas.dataflow"}'
      ',"initCode": true,"wrapperCode": false}'
  ;
run;

Or if the file is large so that converting it to string literals is complicated then use PROC STREAM to process the file and resolve the macro code.

[acordes]

Coming back to my initial problem.

I was confident that executing the sas 'flow' code would fix the issue.

But it doesn't. 

The table gets loaded correctly. 

But then it disappears and the code cannot find the table any more. All these tables disappear from the caslib. 

I attach selected log messages what happens between the successful load and the mysterious disappearance.  

      
1290 +
1291 +data _null_;
1292 +call execute (catx(" " , "proc casutil;", "&str.", "run;"));
NOTE: The quoted string currently being processed has become more than 262 bytes long.  You might have unbalanced quotation marks.
1293 +call execute (catx(" " , "proc casutil;", "&str2.", "run;"));
NOTE: The quoted string currently being processed has become more than 262 bytes long.  You might have unbalanced quotation marks.
1294 +run;
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds
      
NOTE: CALL EXECUTE generated line.
1    + proc casutil;
NOTE: The UUID 'c5290542-dcbb-6f47-82f0-84af2751b792' is connected using session MYSESSION.
1    +               load casdata= 'ATGU_CONFIG_VARIABLES.sashdat' incaslib='public' outcaslib='casuser' casout= 
'ATGU_CONFIG_VARIABLES' ;
NOTE: Cloud Analytic Services made the file ATGU_CONFIG_VARIABLES.sashdat available as table ATGU_CONFIG_VARIABLES in caslib 
      CASUSER(arne.cordes_xyx).
NOTE: The Cloud Analytic Services server processed the request in 0.002227 seconds.

      
1296 +data public.plant_inst;
1297 +set PUBLIC.PLANT_INST;
1298 +where index(combi, "&planta.");
1299 +run;
NOTE: Running DATA step in Cloud Analytic Services.
NOTE: The DATA step will run in multiple threads.
NOTE: There were 4 observations read from the table PLANT_INST in caslib Public.
NOTE: The table plant_inst in caslib Public has 4 observations and 1 variables.
NOTE: DATA statement used (Total process time):
      real time           0.06 seconds
      cpu time            0.03 seconds
      
1300 +
1301 +%put &planta.;
ATGU
1302 +
NOTE: %INCLUDE (level 1) ending.
1303  filename _flw_pr clear;
NOTE: Fileref _FLW_PR has been deassigned.
1304  
1305  /* region: Generated macro cleanup */
1306  %symdel _inputCount;
1307  %symdel _outputCount;
1308  %symdel _input1;
1309  %symdel _output1;
1310  
1311  /* endregion */
1312  
1313  
1314  /* region: Generated step cleanup for Load tables */
1315  %_flw_action_end(id-1729182290124-1493, CASUSER._flw_9731deece3d011ef958c7a00c07, CASUSER, "_flw_9731deece3d011ef958c7a00c07")
1315! ;
_FLW_ACTION_TABLE_|CASUSER|_flw_9731deece3d011ef958c7a00c07|0
_FLW_ACTION_END_|2025-02-05T16:18:06+00:00|id-1729182290124-1493
NOTE: DATA statement used (Total process time):
      real time           0.01 seconds
      cpu time            0.01 seconds
      
1316  /* endregion */
1317  
1318  /*===========================================================================*
1319  * Node name:        Process raw data as of lab data availability and handle exceptions
1320  * Node ID:          id-1732711189188-2066
1321  * Node description: Applies plant and installation specific macro variables.
1322  *                   Requires user knowledge if quality data is available for
1323  *                   plant and installation., see prior prompt.
1324  *
1325  * Input Tables:
1326  *   CASUSER._flw_9731deece3d011ef958c7a00c07
1327  *   CASUSER._flw_9731de39e3d011ef958c7a00c07
1328  *
1329  * Output Tables:
1330  *   CASUSER._flw_9731df27e3d011ef958c7a00c07
1331  *
1332  * Step name:        SAS Program
1333  * Step path:        /dataFlows/steps/a7190700-f59c-4a94-afe2-214ce639fcde
1334  * Step description: Run user written SAS code.
1335  *----------------------------------------------------------------------------*/
1336  
1337  /* region: Generated step setup */
1338  %_flw_action_start(id-1732711189188-2066);
_FLW_ACTION_START_|2025-02-05T16:18:06+00:00|id-1732711189188-2066
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds
      
1339  /* endregion */
1340  
1341  
1342  /* region: Generated macro initialization */
1343  %let _inputCount = 2;
1344  %let _outputCount = 1;
1345  %let _input1 = CASUSER._flw_9731de39e3d011ef958c7a00c07;
1346  %let _input2 = CASUSER._flw_9731deece3d011ef958c7a00c07;
1347  %let _output1 = CASUSER._flw_9731df27e3d011ef958c7a00c07;
1348  
1349  /* endregion */
1350  
1351  FILENAME _flw_pr FILESRVC folderPath="/Public/AC_ODAP/steps and flow code" Filename="process raw data macro template and
1351! exceptions.sas" encoding="UTF-8";
1352  %include _flw_pr / source2;
NOTE: %INCLUDE (level 1) file _FLW_PR is file process raw data macro template and exceptions.sas.
1353 +filename refvars FILESRVC FOLDERPATH='/Public/AC_ODAP/macro_vars_templates/'  FILENAME="&templates..sas";
1354 +
1355 +data _null_;
1356 +%include refvars;
NOTE: %INCLUDE (level 2) file REFVARS is file ATGU_PX10.sas.
1357 +
1358 +%let incas=public; /* caslib for retriving sashdat files
1359 +and due to lack of specific caslibs also destination
1360 +for the report tables */
1361 +
1362 +%let outcas=casuser; /* caslib for the temporary tables during code execution  */
...
1411 +
NOTE: %INCLUDE (level 2) ending.
NOTE: %INCLUDE (level 1) resuming.
1412 +run;
NOTE: DATA statement used (Total process time):
      real time           0.21 seconds
      cpu time            0.06 seconds
      
1413 +
1414 +%put &inst.;
'PX10'
1415 +%put &drops.;
yes
1416 +
1417 +proc sql ;
1418 +    select
1419 +        cats(TAG, '="', Label, '"')
1420 +    into
1421 +        :Column_Labels separated by ' '
1422 +    from
1423 +        &outcas..&planta._CONFIG_VARIABLES
1424 +where Installation in (&inst.);
NOTE: The table CASUSER.ATGU_CONFIG_VARIABLES does not exist in Cloud Analytic Services.
ERROR: File CASUSER.ATGU_CONFIG_VARIABLES.DATA does not exist.
NOTE: PROC SQL set option NOEXEC and will continue to check the syntax of statements.
1425 +quit;
NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE SQL used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds

[acordes]

Just for completeness.

By putting the following initiation at the beginning of the execution, everything runs without problems. 

I don't know why because later in the code and when needed I ensure that all caslibs are assigned. However, now it works.

Thanks to all @Patrick , @Tom , @BrunoMueller and @SASKiwi for pushing me towards the solution.

/* Init CAS library; */
cas mySession sessopts=(caslib=casuser timeout=1800 locale="en_US");
caslib _all_ assign;
cas mySession sessopts=(caslib="casuser");