My VA head node is only seeing one of the worker nodes. The other two are online and were at the time of startup. What is the best way to fix this? I haven't seen this before.
sas.servers stop and start on the VA head node? or the whole environment? or something else?
A little more info: 7 servers in our environment: meta, va-head-node, mid-tier, compute, va worker 1, 2 and 3. Running 9.4 M5 on RHEL environment.
Mine is a gui and it was behind my ssh window.
I ran the code again and I see all three workers now, so I don't know what happened earlier.
@five ,
Please show me the "Last Action Log" from your LASR server. You can get that log in the SAS Visual Analytics Administrator, just do right-click on the LASR server object.
0 -------------- Task Summary -------------- SUCCESS (SASJob PreCode) SUCCESS (Stop Server) -------------- LOG: SASJob PreCode -------------- 1 The SAS System 13:14 Friday, February 14, 2020 NOTE: Copyright (c) 2016 by SAS Institute Inc., Cary, NC, USA. NOTE: SAS (r) Proprietary Software 9.4 (TS1M5) Licensed to *************************************** NOTE: This session is executing on the Linux 2.6.39-400.318.1.el6uek.x86_64 (LIN X64) platform. NOTE: Updated analytical products: SAS/STAT 14.3 SAS/ETS 14.3 SAS/OR 14.3 SAS/IML 14.3 SAS/QC 14.3 WARNING: Your system is scheduled to expire on March 15, 2020, which is 30 days from now. The SAS System will no longer function on or after that date. Please contact your SAS Installation Representative to obtain your updated SAS Installation Data (SID) file, which includes SETINIT information. To locate the name of your SAS Installation Representative go to http://support.sas.com/repfinder and provide your site number ******** and company name as ************************. On the SAS REP list provided, locate the REP for operating system LIN X64. NOTE: Additional host information: Linux LIN X64 2.6.39-400.318.1.el6uek.x86_64 #1 SMP Fri Jan 10 13:33:19 PST 2020 x86_64 Oracle Linux Server release 6.10 You are running SAS 9. Some SAS 8 files will be automatically converted by the V9 engine; others are incompatible. Please see http://support.sas.com/rnd/migration/planning/platform/64bit.html PROC MIGRATE will preserve current SAS file attributes and is recommended for converting all your SAS libraries from any SAS 8 release to SAS 9. For details and examples, please see http://support.sas.com/rnd/migration/index.html This message is contained in the SAS news file, and is presented upon initialization. Edit the file "news" in the "misc/base" directory to display site-specific news and information in the program log. The command line option "-nonews" will prevent this display. NOTE: SAS Initialization used (Total process time): real time 0.00 seconds cpu time 0.00 seconds NOTE: The autoexec file, /opt/sasinside/sasconfig/Lev1/VAApp/WorkspaceServer/autoexec.sas, was executed at server initialization. 1 %let _ENCODING=UTF-8; 2 options VALIDVARNAME=ANY VALIDMEMNAME=EXTEND; 3 /* Status Checkpoint Macro */ 4 %macro statuscheckpoint(maxokstatus=4, varstocheck=SYSERR SYSLIBRC SYSDBRC ); 5 6 %GLOBAL LASTSTEPRC; 7 %LET pos=1; 8 %let var=notset; 9 %let var=%SCAN(&varstocheck.,&pos.); 10 %DO %WHILE ("&VAR." ne ""); 11 /* Retrieve the next return code to check */ 2 The SAS System 13:14 Friday, February 14, 2020 12 %if (%symexist(&VAR.)) %then %do; 13 %let val=&&&VAR..; 14 %if (("&VAL." ne "") and %eval(&VAL. > &maxokstatus.)) %then %do; 15 %put FAIL = &VAR.=&VAL. / SYSCC=&SYSCC.; 16 %let LASTSTEPRC=&VAL.; 17 %end; 18 %end; 19 %let pos = %eval(&pos.+1); 20 %let var=%SCAN(&varstocheck.,&pos.); 21 %END; 22 %mend; -------------- LOG: Stop Server -------------- 3 The SAS System 13:14 Friday, February 14, 2020 23 %statuscheckpoint; 24 /* Skip Next Step If We Have a Bad Status Code */ 25 %macro codeBody; 26 %GLOBAL LASTSTEPRC; 27 %if %symexist(LASTSTEPRC) %then %do; 28 %if %eval(&LASTSTEPRC. <= 4) %then %do; 29 30 /* Stop the server */ 31 proc lasr stop PORT=10031 32 signer="https://*****************:8343/SASLASRAuthorization" 33 ; 34 performance host="wfic-sas-im-hd"; 35 run; 36 37 %end; 38 %end; 39 %mend; 40 %codeBody; WARNING: The SAS LASR Analytic Server product with which LASR is associated will be expiring soon, and is currently in warning mode to indicate this upcoming expiration. Please run PROC SETINIT to obtain more information on your warning period. NOTE: The LASR procedure is executing in the distributed computing environment with 3 worker nodes. NOTE: The LASR Analytic Server on port 10031 of host wfic-sas-im-hd has been terminated. NOTE: The PROCEDURE LASR printed page 1. PROCEDURE| _DISARM| STOP| _DISARM| 2020-02-14T13:14:53,091-05:00| _DISARM| WorkspaceServer| _DISARM| SAS| _DISARM| | _DISARM| 16687104| _DISARM| 16429056| _DISARM| 11| _DISARM| 19| _DISARM| 64| _DISARM| 352| _DISARM| 2.280000| _DISARM| 5.161073| _DISARM| 1897323287.930905| _DISARM| 1897323293.091978| _DISARM| 2.260000| _DISARM| | _ENDDISARM NOTE: PROCEDURE LASR used (Total process time): real time 5.16 seconds cpu time 2.28 seconds -------------- LISTING: Stop Server -------------- The SAS System 13:14 Friday, February 14, 2020 1 The LASR Procedure Performance Information Host Node wfic-sas-im-hd Execution Mode Distributed Number of Compute Nodes 3 -------------- SAS CODE -------------- %let _ENCODING=UTF-8; options VALIDVARNAME=ANY VALIDMEMNAME=EXTEND; /* Status Checkpoint Macro */ %macro statuscheckpoint(maxokstatus=4, varstocheck=SYSERR SYSLIBRC SYSDBRC ); %GLOBAL LASTSTEPRC; %LET pos=1; %let var=notset; %let var=%SCAN(&varstocheck.,&pos.); %DO %WHILE ("&VAR." ne ""); /* Retrieve the next return code to check */ %if (%symexist(&VAR.)) %then %do; %let val=&&&VAR..; %if (("&VAL." ne "") and %eval(&VAL. > &maxokstatus.)) %then %do; %put FAIL = &VAR.=&VAL. / SYSCC=&SYSCC.; %let LASTSTEPRC=&VAL.; %end; %end; %let pos = %eval(&pos.+1); %let var=%SCAN(&varstocheck.,&pos.); %END; %mend; %statuscheckpoint; /* Skip Next Step If We Have a Bad Status Code */ %macro codeBody; %GLOBAL LASTSTEPRC; %if %symexist(LASTSTEPRC) %then %do; %if %eval(&LASTSTEPRC. <= 4) %then %do; /* Stop the server */ proc lasr stop PORT=10031 signer="https://****************:8343/SASLASRAuthorization" ; performance host="*****************"; run; %end; %end; %mend; %codeBody;
The reason that I think we have only one worker in-use is because after running the code found in this thread, I only see one of them in the results and usually I see all three:
@five ,
How many nodes do you see in TKGrid Monitor? You can start the TKGrid monitor using this command: /TKGrid/bin/gridmon.sh.
Not sure if you saw my screenshot, but I saw that as well in the last action log. So I posted why I think we are only seeing one of the workers.
I have not used tkgrid before or setup java before. Give me a few minutes and I'll try to get it working.
Well I got TKGridMon started and it says it's ready on 44346, but I don't know what to do next.
EDIT: Nevermind it was behind my ssh window, I see all three worker nodes. So I am not sure why the report is only showing one of the workers.
@five ,
Depends on the version of TKGrid, you will get a console ( the most recent versions) or GUI application (old versions). The GUI requires an X11 server. Local or remote.
Mine is a gui and it was behind my ssh window.
I ran the code again and I see all three workers now, so I don't know what happened earlier.
@five ,
I'm glad the problem has been resolved. Maybe something was wrong with the network connection?
For others running TKGridMon for the first time, if you get java command not found error, you can edit the gridmon.sh to point to the java location like this:
/opt/sasinside/sashome/SASPrivateJavaRuntimeEnvironment/9.4/jre/bin/java -jar /opt/sasinside/sasva/TKGrid/bin/GridMon.jar Gridhost=$HOSTNAME GridInstall=/opt/sasinside/sasva/TKGrid /opt/sasinside/sasva/TKGrid/bin/tkgridmon
That's what I did at least and it worked. I don't know if that is best practice or not.
Thanks for your help, I learned a few things - so it wasn't a pointless exercise.
Where would that last action log be on the drive itself, instead of going through the GUI?
@five ,
Where would that last action log be on the drive itself, instead of going through the GUI?
You could get the "Last Action Log" only in the SAS VA Administrator.
Join us for SAS Innovate 2025, our biggest and most exciting global event of the year, in Orlando, FL, from May 6-9. Sign up by March 14 for just $795.
See how to use one filter for multiple data sources by mapping your data from SAS’ Alexandria McCall.
Find more tutorials on the SAS Users YouTube channel.