topic Re: Extract Scheduled Jobs from SMC in Developers

Extract Scheduled Jobs from SMC

zdeb15 — Tue, 16 Jul 2019 16:47:49 GMT

I am trying to monitor scheduled jobs in SAS Management Console (SMC). I understand there are dictionaries to pull metadata libraries, views, tables, etc.... ...Is there something similar to pull scheduled Jobs / stored proc?

Re: Extract Scheduled Jobs from SMC

rajdeep — Mon, 12 Aug 2019 12:57:42 GMT

Hi,

If you will use the LSF then you will get a good insight of the same and you can run the below code also to get the scheduled flows. Yes, it's obvious that you need to change the path... or else try to look for something in SMC installation folder and try to replicate the same code.


/*Set this value to the location of your Platform Process Manager installation folder*/
%let pm_install_loc = /sasconfig/pm;

/*Scheduled Flows information. This file seems to be updated by LSF/PM every day at midnight*/
DATA SCHEDULED_FLOWS;
	INFILE "&pm_install_loc./work/system/triggerevents.dat" DSD TRUNCOVER;
	LENGTH TEXT $512. FLOW_OWNER $10. FLOW_NAME $64.;
	INPUT @1 TEXT;	
	IF FIND(TEXT,":") THEN DO;
		FLOW_OWNER = SCAN(TEXT,1,":");	
		FLOW_NAME = SCAN(TEXT,2,":");
	END;
	IF NOT MISSING(FLOW_NAME);
RUN;

/*Flow Information. These files are updated any time a flow is scheduled/rescheduled in SMC or Flow Manager*/
FILENAME XCMD PIPE "ls -ltr &pm_install_loc./work/storage/flow_storage";

/*Create dataset of output from X command*/
DATA FLOW_STORAGE;
	INFILE XCMD DSD TRUNCOVER;
	LENGTH TEXT LOG FILEPATH $512.;
	INPUT @1 TEXT;		
	LOG = SCAN(TEXT,-1," ");	
	IF FIND(TEXT,".dat");
	FILEPATH = "&pm_install_loc./work/storage/flow_storage/"||STRIP(LOG);
	FLOW_OWNER = SCAN(LOG,1,":");
	FLOW_NAME = SCAN(SCAN(LOG,2,":"),1,"%");
	/*Ensuring that only root directory is included in output*/	
	DROP TEXT;
RUN;

Bcoz as of my knowledge I think under the scheduled manager list in SMC, we use to see all the flows which are scheduled, so if you need something else then please explain.

Thanks...

Re: Extract Scheduled Jobs from SMC

sergie89 — Wed, 06 Jan 2021 14:43:17 GMT

@rajdeepIs it also possible to list the type of the trigger in the output list and last modified date of the flow ?

Re: Extract Scheduled Jobs from SMC

MarkBodt_NZ — Wed, 02 Jun 2021 06:04:48 GMT

Hi,

I think rajdeep has pointed you in the direction of the information you are after.

I'll just add a general comment to your question which I interpret as asking is there a way of getting schedule etc information from Metadata?

I haven't attempted to get Job/Deployed Job/Flow etc related metadata, but I have read and written other metadata objects:

Yes, you can read and write metadata programmatically. I wouldn't say it's easy and it takes a bit to get your head around the metadata model.

If you are wanting to access metadata objects, take a look at these documents:

Metadata Model Reference: https://support.sas.com/documentation/cdl/en/omamodref/67417/HTML/default/viewer.htm#titlepage.htm

SAS Language interfaces to metadata https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/lrmeta/titlepage.htm

Mark

Re: Extract Scheduled Jobs from SMC

rajdeep — Sun, 25 Jul 2021 21:01:11 GMT

Hi,

I think you can use the below set of codes to find all Metadata Jobs as well as STPs list. Yeah, if you want any specific metadata folder jobs or STP then have to pass the Type and Attribute variable according to that.

For Jobs:

data work.JOBlist(drop=_: label="SAS JOB List" rename=(name=Job_Name)) ;
  length id $17 _uri name description _modified _created location _location $256;
  length created modified 8;
  format created modified datetime.;
  label id="Metadata ID"
        name="JOB Name"
        description="Description"
        location="Folder Location"
        created="Created"
        modified="Last Modified";
  _nobj=1;
  _n=1;
  call missing(id, _uri, name, description, _modified, _created, _location);
 
  do while(_n le _nobj);
    _nobj=metadata_getnobj("omsobj:Job?@Id contains '.'",_n,_uri);

    _rc=metadata_getattr(_uri,"Id",id);
    _rc=metadata_getattr(_uri,"Name",name);
    _rc=metadata_getattr(_uri,"Desc",description);
    _rc=metadata_getattr(_uri,"MetadataCreated",_created);
    _rc=metadata_getattr(_uri,"MetadataUpdated",_modified);
 
    created=input(_created,anydtdtm.);
    modified=input(_modified,anydtdtm.);
 
    * Get folder object the current JOB is in *;
    _rc=metadata_getnasn(_uri,"Trees",1,_uri);
    * Get folder name the current JOB is in *;
    _rc=metadata_getattr(_uri,"Name",location);
    _tree=1;
    * Loop up the folder hierarchy *;
    do while (_tree>0);
      * Get the parent folder object *;
      _tree=metadata_getnasn(_uri,"ParentTree",1,_uri);
      if _tree > 0 then do;
        * If there was a parent folder, get the name *;
        _rc=metadata_getattr(_uri,"Name",_location);
        * Construct the path *;
        location=catx('/',_location,location);
      end;
    end; * Folder Hierachy *;
     location = '/'||location;
    output;
    _n=_n+1;
  end;
run;

For STPs:

data work.stplist(drop=_: label="SAS Stored Process List");
  length id $17 _uri name description _modified _created location _location $256;
  length created modified 8;
  format created modified datetime.;
  label id="Metadata ID"
        name="Stored Process Name"
        description="Description"
        location="Folder Location"
        created="Created"
        modified="Last Modified";
  _nobj=1;
  _n=1;
  call missing(id, _uri, name, description, _modified, _created, _location);
 
  do while(_n le _nobj);
    _nobj=metadata_getnobj("omsobj:ClassifierMap?@PublicType='StoredProcess'",_n,_uri);
    _rc=metadata_getattr(_uri,"Id",id);
    _rc=metadata_getattr(_uri,"Name",name);
    _rc=metadata_getattr(_uri,"Desc",description);
    _rc=metadata_getattr(_uri,"MetadataCreated",_created);
    _rc=metadata_getattr(_uri,"MetadataUpdated",_modified);
 
    created=input(_created,anydtdtm.);
    modified=input(_modified,anydtdtm.);
 
    * Get folder object the current STP is in *;
    _rc=metadata_getnasn(_uri,"Trees",1,_uri);
    * Get folder name the current STP is in *;
    _rc=metadata_getattr(_uri,"Name",location);
    _tree=1;
    * Loop up the folder hierarchy *;
    do while (_tree>0);
      * Get the parent folder object *;
      _tree=metadata_getnasn(_uri,"ParentTree",1,_uri);
      if _tree > 0 then do;
        * If there was a parent folder, get the name *;
        _rc=metadata_getattr(_uri,"Name",_location);
        * Construct the path *;
        location=catx('/',_location,location);
      end;
    end; * Folder Hierachy *;
     location = '/'||location;
     output;
    _n=_n+1;
  end;
run;

Thanks.

Re: Extract Scheduled Jobs from SMC

AllanBowe — Mon, 26 Jul 2021 07:50:20 GMT

Yes - there's a macro for it in the SASjs/core library: https://core.sasjs.io/mm__tree_8sas.html

To fetch all Deployed Jobs and STPs in one go:

/* load macros */
filename mc url  "https://raw.githubusercontent.com/sasjs/core/main/all.sas";
%inc mc;

/* export only jobs and STPs */
%mm_tree(root=/
  ,types=DeployedJob StoredProcess
  ,outds=work.mycontent
)

Re: Extract Scheduled Jobs from SMC

zdeb15 — Tue, 27 Jul 2021 12:45:25 GMT

This is exactly what I needed, thank you! Do you know if I can pull the actual schedule also? Ex: Run 1x Daily at 5am.

Re: Extract Scheduled Jobs from SMC

sergie89 — Tue, 27 Jul 2021 13:23:55 GMT

Hello @rajdeep ,

Thanks for your response. Is it also possible to this for the scheduled flows ?

Re: Extract Scheduled Jobs from SMC

rajdeep — Tue, 27 Jul 2021 14:31:31 GMT

Hi sergie89,

The code what I posted above was for all jobs that are available in Metadata, but if you need the list for only Deployed jobs or any flow if you had scheduled in SMC over Metadata then you just need to change the below from the above code.

_nobj=metadata_getnobj("omsobj:JOB?@Id contains '.'",_n,_uri);

_nobj=metadata_getnobj("omsobj:JFJOB?@Id contains '.'",_n,_uri);

Try that, I think it will just list out the flow names and deployed jobs. If anything else let me know I will also try.

Re: Extract Scheduled Jobs from SMC

zdeb15 — Tue, 27 Jul 2021 14:33:35 GMT

This works but more particular I am looking for the information related to the flow, under scheduler manager option in SAS Management Console. I am thinking since this information gets stored in windows task manager, I would have to link the and the flow to the scheduled tasks from task manager? I am not 100% sure if this is stored in metadata.

Re: Extract Scheduled Jobs from SMC

rajdeep — Tue, 27 Jul 2021 14:35:42 GMT

Hi zdeb15,

Just want to know which flow manager you are using for your environment to schedule the jobs. If it's LSF then I have posted the code related to it in the past reply chain.

Also, please let me know what you need as an output table columns exactly.

Thanks.

Re: Extract Scheduled Jobs from SMC

rajdeep — Tue, 27 Jul 2021 15:49:48 GMT

Hi zdeb15,

I would suggest go to the C:\Windows\System32\Tasks directory, there you can retrieve the schedule details regarding the flows from SMC.
Yeah, it will be a XML version file, but through a small program we can pull the required parameters and it's values into a structured fashion.

From metadata information point of view I tried, but it seems timeline and interval related attributes are not getting stored inside Metadata. But if I get any info regarding this will post here.

Thanks in advance.

Re: Extract Scheduled Jobs from SMC

kevin_stanford — Mon, 29 Aug 2022 18:04:09 GMT

I am looking for the information related to the flow, under scheduler manager option in SAS Management Console.

Were you ever able to find this information in metadata?