topic Re: Question related to DataMart’s of SAS Environment Manager ( version 2.5.1) in SAS Programming

Question related to DataMart’s of SAS Environment Manager ( version 2.5.1)

mmkr — Thu, 09 Apr 2020 21:45:22 GMT

Hi all,

Currently artifactusagedetails datamart is updating once in every 24 hours

Can we make updates to have real-time, if so how can we achieve that.

Thanks

Mohan.

Re: Question related to DataMart’s of SAS Environment Manager ( version 2.5.1)

AnandVyas — Fri, 10 Apr 2020 06:07:13 GMT

Hi Mohan,

As you might be aware that there is an ETL process for APM log collection (after the SAS service architecture has been initialized) that runs every night to collect the logs locally (If you have a Multi-Tier architecture) and shares it to a central "Landing Zone" where they are parsed and data is updated in the EVM data-mart. Below is the diagram of the flow.

This process usually runs at 1 AM which is after the rolling of logs for most of the SAS Services happen. You can run this ETL on a adhoc basis using the SAS EVM application by following the process below.

Manually Discovering and Collecting Logs

Follow these steps to manually run the APM ETL process to discover and collect logs:

Select Resources
->

Browse

->
Services. Select the APM ETL Processing entry in the table.
On the page for the APM ETL Processing service, select Control.

In the Quick Control area, select Run in the Control Action field and then click the Run icon

You can also schedule to run this ETL process using command line script "master_apm_etl.sh" which is located by default at location "<SAS-Configuration>/Lev1/Web/SASEnvironmentManager/emi-framework/bin/"

In my view depending upon the architecture at your site there could be a lot of log information generated and running the ETL process every min (to achieve real time) could be a lot of house-keeping overhead instead and vice-versa if there is no much activity on the platform then there would no data. So I think a balanced time has to be chosen based on the usage. SAS documentation also suggest to run this every 30 mins to get the view of the information stored in the logs.

You can find more information about this on the links:

Working with APM ETL: https://go.documentation.sas.com/?docsetId=evug&docsetTarget=n0ejv8qtopfvman1v14k19ejxa4u.htm&docsetVersion=2.5&locale=en

Running the script manually: https://go.documentation.sas.com/?docsetId=evug&docsetTarget=p0y1q4ja8r14d1n1j2yzp4h0mqa8.htm&docsetVersion=2.5&locale=en#n1p03ojzd6t08gn1spuc2lwfmtwx

Hope this helps!

Re: Question related to DataMart’s of SAS Environment Manager ( version 2.5.1)

mmkr — Mon, 13 Apr 2020 09:09:47 GMT

Hi Anand

Thanks you so much for detailed response.

Our server is in Windows Environment.

all i see is these four datamarts (ACM, ARTIFACT, EVDM & KITS ) in my log in.

Do we need to enable/activate APM ETL ?

In ARTIFACT i can see logs processed ( we have Logfile Parse Timestamp this is also updated once in 24 hours).

Kind Note:

Actually i am SAS programmer trying to see main contributor of SERVER slowness. Please excuse me if i am asking to many things :-).

Please advise.

Thanks,

Re: Question related to DataMart’s of SAS Environment Manager ( version 2.5.1)

AnandVyas — Mon, 13 Apr 2020 09:44:34 GMT

Only after the APM ETL is enabled in the system, all these data is collected.
You can see this when you login because these are pre-assigned libraries pointing to physical location that these data-sets are saved at.

Re: Question related to DataMart’s of SAS Environment Manager ( version 2.5.1)

mmkr — Thu, 16 Apr 2020 11:39:44 GMT

Thanks a lot Anand..

what would be the Datamart name if APM ETL is activated.

at my login in SAS EG i can see only ACM,ARTIFACT,EVDM,KITS .. We should have APM as datamart if APM ETL activated..

Please advice.

Re: Question related to DataMart’s of SAS Environment Manager ( version 2.5.1)

AnandVyas — Thu, 16 Apr 2020 13:43:38 GMT

Those tables are located inside sub-folders under a file system path called datamart.
Datamart path - <config>/Lev1/Web/SASEnvironmentManager/emi-framework/Datamart

For Ex: ACM path would be <config>/Lev1/Web/SASEnvironmentManager/emi-framework/Datamart/acm
Ref: https://support.sas.com/documentation/cdl/en/evug/67886/HTML/default/viewer.htm#n13wvzy0q896tsn19cqmssxv1pza.htm

Re: Question related to DataMart’s of SAS Environment Manager ( version 2.5.1)

mmkr — Thu, 16 Apr 2020 18:29:47 GMT

Thanks Anad.. Will check that..