topic Re: SAS Flow Manager in Administration and Deployment

SAS Flow Manager

jogendra — Tue, 12 May 2020 21:14:14 GMT

I am New to SAS and wanted to check if I can export all the flows which are running in the flow manager to an excel sheet. Also I wanted to see where my JOb execution is stored. After a job completed where is the completion time and date stored for the job.

Please help

Re: SAS Flow Manager

LinusH — Wed, 13 May 2020 11:01:24 GMT

Do you refer to the IBM/Platform client?

This software is bundled with SAS, but it's not a SAS sw. Detailed queston on this may be found here:

https://www.ibm.com/support/knowledgecenter/SSZSHQ_10.2.0/source/pm_applications.html

By "where my JOb execution is stored", do you mean like in a log database table?

On the SAS side you can see execution results in form of log files, and potentially in Environment Manager if you have that configured for your batches.

Re: SAS Flow Manager

jogendra — Wed, 13 May 2020 12:07:52 GMT

I wanted to copy of all my flow or Job running in the schedule for a day. I need then to be downloaded and store in a excels sheet.

Also I need to download the run history of each job. to calculate the time take by a job complete.

Re: SAS Flow Manager

gwootton — Wed, 13 May 2020 13:46:26 GMT

Hi @jogendra

Flows scheduled in SAS Management Console have schedules stored in Metadata. I posted a program here that pulls this information into a data set:

https://communities.sas.com/t5/Administration-and-Deployment/How-to-know-the-batch-jobs-scheduled-times/td-p/640462

Flow history for the Process Manager component of Platform Suite for SAS are captured in files contained in JS_TOP/work/history/ (where JS_TOP is the Process Manager installation directory). This SAS program is an example of how you could parse these files into SAS:

/* Define the path to the history files. */
filename history "/sas94/pss/pm/work/history/*";

%let ndays = 14;
options ls = 150;

data flows;
   length username $ 16 flowname jobname $ 32 flowid jobid $ 6 state $ 12 started ended 8;
   keep   username      flowname jobname      flowid jobid     state      started ended ;
   length stats    $ 64 flowjob  $ 200;
   format started ended datetime19.;
   retain cutoff 0;
   if _n_ = 1 then cutoff = tintnx('DTday',datetime(),-&ndays,'b'); /* midnight one week ago */
   infile history;
   input;
   timestamp = tintnx('DTyear',scan(_infile_,5,'"'),10,'s');
   if timestamp < cutoff then delete;
   if not(index(_infile_,"Finished ")) then delete;
   flowjob = scan(_infile_,7,'"');
   username = scan(_infile_,3,'"');
   flowid   = scan(flowjob,1,':');
   flowname = scan(flowjob,3,':');
   jobname  = scan(flowjob,4,':{');
   stats    = scan(_infile_,11,'"');
   if _infile_ =: '"FLOW"' then
      do;
         state   = scan(stats,2,'=|')||'('||scan(stats,4,'=|')||')';
         started = tintnx('DTyear',scan(stats,6,'=|'),10,'s');
         ended   = timestamp;
         jobid = " ";
      end;
   else
   if _infile_ =: '"JOB"' then
      do;
         jobid   = scan(stats,2,'=|');
         state   = scan(stats,4,'=|')||'('||scan(stats,6,'=|')||')';
         started = tintnx('DTyear',scan(stats,8,'=|'),10,'s');
         ended   = timestamp;
      end;
run;
proc sort;  by flowid jobid; run; 
proc print; by flowid flowname;  id flowid flowname;  var jobid jobname username started ended state;  run;

Re: SAS Flow Manager

rldaven — Thu, 31 Dec 2020 19:02:05 GMT

Do you happen to have any examples that would include the Host name used for the job flow execution in addition to the timestamps, etc?

Re: SAS Flow Manager

gwootton — Mon, 04 Jan 2021 14:47:40 GMT

I do not have any additional examples for reading the history file. This code only reads from lines that contain the word "Finished":
if not(index(_infile_,"Finished ")) then delete;

The execution host is not contained on the Finished line but on the Execute line:
"JOB" "sas" "1603822696" "193:sas:sysget_flow:sysget" "Execute job" "JobId=31231|Host=trcv051.trc.sas.com"
"JOB" "sas" "1603822701" "193:sas:sysget_flow:sysget" "Finished job" "JobId=31231|State=Done|Status=0|StartTime=1603822696|FinishTime=1603822701|CPUUsage=0.346100 sec"

The easiest way I can think of would be to modify the code to read the execute lines into a separate lookup table (job id / host) and then join that to the table created by the existing program.