I have an open tech support track on this but wanted to ask here as well. One of our data scientists is working on SAS Code which uses rsubmit and has run into an issue where it works in Prod but not Dev. He gave me simplified code that demonstrates the issue and I'm able to repeat it with my credentials. My searches for the errors I'm encountering have not thus far yielded information that assists, e.g. I got a SAS note about a hotfix for 9.4M0.
The environments are SAS 9.4M7 with hotfixes current as of the time of their upgrades (mid-December of last year for Dev and February of this year for Prod.) The grid is SAS Workload Orchestrator rather than LSF. Integrated Windows Authentication (IWA) is used for most client access, particularly via Enterprise Guide. The environments were installed several months apart over a year ago. I was not on this client when either install was done but was for the upgrades. I have observed some configuration differences between environments such as log locations and file system layouts on certain nodes (particularly midtier.)
The two data files are trivial example data; both are two columns with 3713 rows. The columns are "date" which contains dates in ddMMMyyyy format starting with 01JUL2020. The task column has the same value for every row; 1 in "task1_data" and 2 in "task2_data."
So far with SAS Technical Support the focus has been on the metadata identity having an external account and password provided. By default the account tab has the username but not the password since SAS is configured to use PAM, and the metadata identities are created the same in both environments. As a diagnostic step I added my password to the external credentials in my Dev metadata identity then started a new session in EG without any change in behavior.
The demo code (note: edited to remove my username.)
LIBNAME bee BASE "/sasdata/sasusers/username/BEE";
data work.dates ;
format date date10.;
do date=22097 to 22159+(365*10);
output;
end;
run;
/*start rsubmit code*/
/*%let rc=%sysfunc(grdsvc_enable(_all_,server=SASApp));*/
%let rc=%sysfunc(grdsvc_enable(_all_, resource=SASApp));
options autosignon;
libname shared "%sysfunc(pathname(work))";
rsubmit task1 wait=no inheritlib=(shared);
LIBNAME bee BASE "/sasdata/sasusers/username/BEE";
data bee.task1_data ;
set shared.dates;
task=1;
run;
endrsubmit;
rsubmit task2 wait=no inheritlib=(shared);
LIBNAME bee BASE "/sasdata/sasusers/username/BEE";
data bee.task2_data ;
set shared.dates;
task=2;
run;
endrsubmit;
waitfor _all_ task1 task2;
signoff _all_;
data work.shared_combined;
set bee.task2_data bee.task2_data;
run;
This is the log (with replacements of username and identifying hostnames)
1 The SAS System Thursday, May 13, 2021 11:38:00 AM
1 ;*';*";*/;quit;run;
2 OPTIONS PAGENO=MIN;
3 %LET _CLIENTTASKLABEL='rsubmit_example.sas';
4 %LET _CLIENTPROCESSFLOWNAME='Standalone Not In Project';
5 %LET _CLIENTPROJECTPATH='';
6 %LET _CLIENTPROJECTPATHHOST='';
7 %LET _CLIENTPROJECTNAME='';
8 %LET _SASPROGRAMFILE='C:\Users\username\Documents\rsubmit_example.sas';
9 %LET _SASPROGRAMFILEHOST='laptop_24558';
10
11 ODS _ALL_ CLOSE;
12 OPTIONS DEV=SVG;
13 GOPTIONS XPIXELS=0 YPIXELS=0;
14 %macro HTML5AccessibleGraphSupported;
15 %if %_SAS_VERCOMP_FV(9,4,4, 0,0,0) >= 0 %then ACCESSIBLE_GRAPH;
16 %mend;
17 FILENAME EGHTML TEMP;
18 ODS HTML5(ID=EGHTML) FILE=EGHTML
19 OPTIONS(BITMAP_MODE='INLINE')
20 %HTML5AccessibleGraphSupported
21 ENCODING='utf-8'
22 STYLE=HTMLBlue
23 NOGTITLE
24 NOGFOOTNOTE
25 GPATH=&sasworklocation
26 ;
NOTE: Writing HTML5(EGHTML) Body file: EGHTML
27
28 LIBNAME bee BASE "/sasdata/sasusers/username/BEE";
NOTE: Libref BEE was successfully assigned as follows:
Engine: BASE
Physical Name: /sasdata/sasusers/username/BEE
29
30 data work.dates ;
31 format date date10.;
32 do date=22097 to 22159+(365*10);
33 output;
34 end;
35 run;
NOTE: The data set WORK.DATES has 3713 observations and 1 variables.
NOTE: MVA_DSIO.OPEN_CLOSE| _DISARM| STOP| _DISARM| 2021-05-13T11:38:23,057-05:00| _DISARM| WorkspaceServer| _DISARM| |
_DISARM| | _DISARM| | _DISARM| 19820544| _DISARM| 13| _DISARM| 14| _DISARM| 136| _DISARM| 1040| _DISARM| | _DISARM| |
_DISARM| | _DISARM| | _DISARM| | _DISARM| | _ENDDISARM
NOTE: PROCEDURE| _DISARM| STOP| _DISARM| 2021-05-13T11:38:23,058-05:00| _DISARM| WorkspaceServer| _DISARM| | _DISARM| |
_DISARM| | _DISARM| 20082688| _DISARM| 13| _DISARM| 14| _DISARM| 136| _DISARM| 1040| _DISARM| | _DISARM| | _DISARM| |
_DISARM| | _DISARM| | _DISARM| | _ENDDISARM
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds
36
37
38
39 /*start rsubmit code*/
40 /*%let rc=%sysfunc(grdsvc_enable(_all_,server=SASApp));*/
41 %let rc=%sysfunc(grdsvc_enable(_all_, resource=SASApp));
2 The SAS System Thursday, May 13, 2021 11:38:00 AM
42
43 options autosignon;
44
45 libname shared "%sysfunc(pathname(work))";
NOTE: Libref SHARED refers to the same physical library as WORK.
NOTE: Libref SHARED was successfully assigned as follows:
Engine: V9
Physical Name:
/saswork/SAS_work58100000148C_gridhost2.example.com/SAS_workB2EA0000148C_gridhost2.example.com
46
47
48
49 rsubmit task1 wait=no inheritlib=(shared);
NOTE: Remote session ID TASK1 will use the grid service _ALL_.
NOTE: Remote signon to TASK1 commencing (SAS Release 9.04.01M7P080520).
ERROR: PROVIDER(SAS): Unable to get grid information from the master host.
ERROR: Cannot initialize the grid provider module.
ERROR: A communication subsystem partner link setup request failure has occurred.
ERROR: Cannot initialize the grid provider module.
NOTE: SAS.Connect.SIGNON| _DISARM| STOP| _DISARM| 2021-05-13T11:38:23,203-05:00| _DISARM| SAS/CONNECT:TASK1| _DISARM| SAS|
_DISARM| | _DISARM| | _DISARM| | _DISARM| | _DISARM| | _DISARM| | _DISARM| | _DISARM| 0.010000| _DISARM| 0.040638| _DISARM|
1936543103.162876| _DISARM| 1936543103.203514| _DISARM| 0.010000| _DISARM| | _ENDDISARM
NOTE: Subsequent lines will be ignored until ENDRSUBMIT.
ERROR: Remote submit to TASK1 canceled.
50
51
52
53 rsubmit task2 wait=no inheritlib=(shared);
NOTE: Remote session ID TASK2 will use the grid service _ALL_.
NOTE: Remote signon to TASK2 commencing (SAS Release 9.04.01M7P080520).
ERROR: PROVIDER(SAS): Unable to get grid information from the master host.
ERROR: Cannot initialize the grid provider module.
ERROR: A communication subsystem partner link setup request failure has occurred.
ERROR: Cannot initialize the grid provider module.
NOTE: SAS.Connect.SIGNON| _DISARM| STOP| _DISARM| 2021-05-13T11:38:23,233-05:00| _DISARM| SAS/CONNECT:TASK2| _DISARM| SAS|
_DISARM| | _DISARM| | _DISARM| | _DISARM| | _DISARM| | _DISARM| | _DISARM| | _DISARM| 0.000000| _DISARM| 0.023485| _DISARM|
1936543103.210368| _DISARM| 1936543103.233853| _DISARM| 0.000000| _DISARM| | _ENDDISARM
NOTE: Subsequent lines will be ignored until ENDRSUBMIT.
ERROR: Remote submit to TASK2 canceled.
NOTE: There are no active tasks/transactions.
54
55 waitfor _all_ task1 task2;
56 signoff _all_;
NOTE: No active remote sessions exist; SIGNOFF is not necessary.
57
58
59
60 data work.shared_combined;
61 set bee.task2_data bee.task2_data;
62 run;
NOTE: There were 3713 observations read from the data set BEE.TASK2_DATA.
NOTE: MVA_DSIO.OPEN_CLOSE| _DISARM| STOP| _DISARM| 2021-05-13T11:38:23,250-05:00| _DISARM| WorkspaceServer| _DISARM| |
_DISARM| | _DISARM| | _DISARM| 20344832| _DISARM| 13| _DISARM| 17| _DISARM| 136| _DISARM| 1176| _DISARM| | _DISARM| |
_DISARM| | _DISARM| | _DISARM| | _DISARM| | _ENDDISARM
NOTE: There were 3713 observations read from the data set BEE.TASK2_DATA.
NOTE: MVA_DSIO.OPEN_CLOSE| _DISARM| STOP| _DISARM| 2021-05-13T11:38:23,250-05:00| _DISARM| WorkspaceServer| _DISARM| |
_DISARM| | _DISARM| | _DISARM| 20344832| _DISARM| 13| _DISARM| 17| _DISARM| 136| _DISARM| 1176| _DISARM| | _DISARM| |
_DISARM| | _DISARM| | _DISARM| | _DISARM| | _ENDDISARM
NOTE: The data set WORK.SHARED_COMBINED has 7426 observations and 2 variables.
NOTE: MVA_DSIO.OPEN_CLOSE| _DISARM| STOP| _DISARM| 2021-05-13T11:38:23,251-05:00| _DISARM| WorkspaceServer| _DISARM| |
_DISARM| | _DISARM| | _DISARM| 20344832| _DISARM| 13| _DISARM| 17| _DISARM| 264| _DISARM| 1304| _DISARM| | _DISARM| |
_DISARM| | _DISARM| | _DISARM| | _DISARM| | _ENDDISARM
NOTE: PROCEDURE| _DISARM| STOP| _DISARM| 2021-05-13T11:38:23,251-05:00| _DISARM| WorkspaceServer| _DISARM| | _DISARM| |
3 The SAS System Thursday, May 13, 2021 11:38:00 AM
_DISARM| | _DISARM| 20344832| _DISARM| 13| _DISARM| 17| _DISARM| 264| _DISARM| 1304| _DISARM| | _DISARM| | _DISARM| |
_DISARM| | _DISARM| | _DISARM| | _ENDDISARM
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.01 seconds
63
64 %LET _CLIENTTASKLABEL=;
65 %LET _CLIENTPROCESSFLOWNAME=;
66 %LET _CLIENTPROJECTPATH=;
67 %LET _CLIENTPROJECTPATHHOST=;
68 %LET _CLIENTPROJECTNAME=;
69 %LET _SASPROGRAMFILE=;
70 %LET _SASPROGRAMFILEHOST=;
71
72 ;*';*";*/;quit;run;
73 ODS _ALL_ CLOSE;
74
75
76 QUIT; RUN;
77
