Rolling log option is getting reverted to '_' from '_#Y.#m.#d_#H.#M.#s.log' every night.

[Rajkumar20]

Hi All,

As a part of migration project testing, we are trying to modify the rolling log options in app server (SMC) to '_#Y.#m.#d_#H.#M.#s.log' from '_'.

But  rolling log options is getting reverted back to '_' every night. Please let me know if there is any other config files to be changed.

Regards,

Rajkumar.

[gwootton]

Hi Rajkumar!

This is a function of the SAS Environment Manager Service Architecture Framework. When enabled, the logDiscovery job sets this value to "_" in SAS Management Console so when jobs are deployed their -LOG option is specified as the log path followed by the job name and _.

This is then picked up by the SASApp/BatchServer/logconfig.apm.xml, which appends the date and time to the end of what is defined in the -LOG option using the %S{App.Log} conversion character:

<param name="FileNamePattern" value="%S{App.Log}%d{yyyy.MM.dd_hh.mm.ss}.log"/>

You can confirm this by looking in the logDiscovery logs in <SAS-configuration-directory>/Web/SASEnvironmentManager/emi-client/Logs/logDiscovery_date_time.log, it will contain this stanza referencing the change.

2020-03-13 02:10:35,752 INFO [main] [configureAPM] - ****************************
2020-03-13 02:10:35,752 INFO [main] [configureAPM] - Found 1 'Server.DataStepBatch' objects in metadata
2020-03-13 02:10:35,769 INFO [main] [configureAPM] - No Update: property value 'BatchServer.DataStep.Property.LogExt.xmlKey.txt' is correct
2020-03-13 02:10:35,770 INFO [main] [configureAPM] - Existing value is '_'

[Rajkumar20]

Thanks much.

I can see the below in the logs. Please let me know how I can change the rolling log options as '_#Y.#m.#d_#H.#M.#s.log'  and stop it reverting back to '_'.

2020-03-13 09:05:15,872 INFO  [main] [configureAPM] - Found 3 'LogicalServer.PooledWorkspace' objects in metadata
2020-03-13 09:05:15,969 INFO  [main] [configureAPM] - No Update: property value 'Initialization program' is correct
2020-03-13 09:05:15,969 INFO  [main] [configureAPM] - Existing value is 'SASEnvironment/SASMacro/apm_request_init.sas'

2020-03-13 09:05:15,971 INFO  [main] [configureAPM] - No Update: property value 'Termination program' is correct
2020-03-13 09:05:15,971 INFO  [main] [configureAPM] - Existing value is 'SASEnvironment/SASMacro/apm_request_term.sas'

2020-03-13 09:05:15,972 INFO  [main] [configureAPM] - No Update: property value 'Initialization program' is correct
2020-03-13 09:05:15,972 INFO  [main] [configureAPM] - Existing value is 'SASEnvironment/SASMacro/apm_request_init.sas'

2020-03-13 09:05:15,973 INFO  [main] [configureAPM] - No Update: property value 'Termination program' is correct
2020-03-13 09:05:15,973 INFO  [main] [configureAPM] - Existing value is 'SASEnvironment/SASMacro/apm_request_term.sas'

2020-03-13 09:05:15,974 INFO  [main] [configureAPM] - No Update: property value 'Initialization program' is correct
2020-03-13 09:05:15,974 INFO  [main] [configureAPM] - Existing value is 'SASEnvironment/SASMacro/apm_request_init.sas'

2020-03-13 09:05:15,975 INFO  [main] [configureAPM] - No Update: property value 'Termination program' is correct
2020-03-13 09:05:15,975 INFO  [main] [configureAPM] - Existing value is 'SASEnvironment/SASMacro/apm_request_term.sas'

2020-03-13 09:05:16,038 INFO  [main] [configureAPM] - ****************************
2020-03-13 09:05:16,038 INFO  [main] [configureAPM] - Found 3 'LogicalServer.StoredProcess' objects in metadata
2020-03-13 09:05:16,046 INFO  [main] [configureAPM] - No Update: property value 'Initialization program' is correct
2020-03-13 09:05:16,046 INFO  [main] [configureAPM] - Existing value is 'SASEnvironment/SASMacro/apm_request_init.sas'

2020-03-13 09:05:16,047 INFO  [main] [configureAPM] - No Update: property value 'Termination program' is correct
2020-03-13 09:05:16,047 INFO  [main] [configureAPM] - Existing value is 'SASEnvironment/SASMacro/apm_request_term.sas'

2020-03-13 09:05:16,049 INFO  [main] [configureAPM] - No Update: property value 'Initialization program' is correct
2020-03-13 09:05:16,049 INFO  [main] [configureAPM] - Existing value is 'SASEnvironment/SASMacro/apm_request_init.sas'

2020-03-13 09:05:16,050 INFO  [main] [configureAPM] - No Update: property value 'Termination program' is correct
2020-03-13 09:05:16,050 INFO  [main] [configureAPM] - Existing value is 'SASEnvironment/SASMacro/apm_request_term.sas'

2020-03-13 09:05:16,060 INFO  [main] [configureAPM] - No Update: property value 'Initialization program' is correct
2020-03-13 09:05:16,060 INFO  [main] [configureAPM] - Existing value is 'SASEnvironment/SASMacro/apm_request_init.sas'

2020-03-13 09:05:16,061 INFO  [main] [configureAPM] - No Update: property value 'Termination program' is correct
2020-03-13 09:05:16,062 INFO  [main] [configureAPM] - Existing value is 'SASEnvironment/SASMacro/apm_request_term.sas'

2020-03-13 09:05:16,160 INFO  [main] [configureAPM] - ****************************
2020-03-13 09:05:16,160 INFO  [main] [configureAPM] - Found 2 'Server.DataStepBatch' objects in metadata
2020-03-13 09:05:16,167 INFO  [main] [configureAPM] - No Update: property value 'BatchServer.DataStep.Property.LogExt.xmlKey.txt' is correct
2020-03-13 09:05:16,167 INFO  [main] [configureAPM] - Existing value is '_'

2020-03-13 09:05:16,168 INFO  [main] [configureAPM] - No Update: property value 'BatchServer.DataStep.Property.LogExt.xmlKey.txt' is correct
2020-03-13 09:05:16,168 INFO  [main] [configureAPM] - Existing value is '_'

Regards,

Rajkumar.

[gwootton]

Thanks Rajkumar,

This setting is expected for the Service Architecture Framework to function, are you sure you want to change it? What is the problem you are trying to solve?

Disabling the Service Architecture Framework would prevent the logDiscovery script from modifying this setting. The documentation for doing this can be found here:

SAS® Environment Manager 2.5: User’s Guide - Disabling the ACM and APM ETL Processes 

[Rajkumar20]

When we submit a batch job, we need the log file to be generated as following,

For example, a Job 'XYZ' submitted today(16th March 2020) at 10.00 AM.

Expected to be created as: XYZ_2020.03.16_10.00.00.log

instead the log file is generated as below

XYZ_#Y.#m.#d_#H.#M.#s.log2020.03.16_10.00.00.log

Please suggest us a way to generate the log file as expected.

[gwootton]

Thanks Rajkumar,

That would occur if you change the rolling log option from "_" to "_#Y.#m.#d_#H.#M.#s.log" then deploying the job while that setting was present, without making any changes to the logconfig file. The logconfig.apm.xml file takes the contents of the -LOG option, which would be something along the lines of  "/path/jobname_#Y.#m.#d_#H.#M.#s.log" and appends the date and time to that.

Setting the rolling log option back to "_" and redeploying the job should have the log generated as expected.

[Rajkumar20]

Hi,

Rolling log option in SAS Manangement console is set as '_'. Please find attached 'logconfig.apm.xml' & 'logconfig.xml' files.

But still the log file is getting generated in the same format.

Please let me know if any changes to be done in the config files.

[gwootton]

Thanks Rajkumar,

After reverting the configuration did you redeploy the job in question? In the properties for the deployed job in SAS Management Console under the Scheduling Details tab it should list the command. Confirm the -LOG option specifies jobname_ rather than jobname_#Y.#m.#d_#H.#M.#s.log:

<SAS-Configuration-Directory>/Levn/SASApp/BatchServer/sasbatch.sh -log <SAS-Configuration-Directory>/Levn/SASApp/BatchServer/Logs/job_name_ ...

If it specifies #Y.#m.#d_#H.#M.#s.log, you should redeploy the job. This can be done by right-clicking the Schedule Manager plug-in and selecting "Manage Deployed Jobs", selecting the job in question and clicking "Redeploy". 

This will reset the "command" 

[Rajkumar20]

Thanks much.

Please find attached the steps while redeploying jobs.

Logs has been created as expected in the lower environments only when command line contains '_#Y.#m.#d_#H.#M.#s.log' but not in the reported case.

In the issue reported environment, log is created as expected without this '_#Y.#m.#d_#H.#M.#s.log' with the command line.

Please let us know how this can be rectified in lower environments too.

[gwootton]

Thanks Rajkumar,

The addition of the date is performed by sourcing the DATA Step Batch Server's logconfig file (either logconfig.xml or logconfig.apm.xml depending on configuration) when the parameter "_" is used and the date is present. If you are saying that in other environments you must have the '_#Y.#m.#d_#H.#M.#s.log' line, then either a logconfig file is not being called in that environment at all or is not being used to append the date to the LOG option.

For example, in the logconfig files you provided the FileNamePattern is set to:

<param name="FileNamePattern" value="%S{App.Log}%d{yyyy.MM.dd_hh.mm.ss}.log"/>

The value %S{App.Log} translates to "whatever the -LOG option was set to" and %d{yyyy.MM.dd_hh.mm.ss} translates as the date and time in that supplied order, so if the -LOG option was set to /path/to/job_name_ the log file would be /path/to/job_name_2020.03.24_12.01.00.log.

SAS® 9.4 Logging: Configuration and Programming Reference - S Conversion Character

SAS® 9.4 Logging: Configuration and Programming Reference - d Conversion Character

[FK1]

Hi @gwootton ,

I do have a similar problem.

When running a job, the name of the logfile looks like this:

Test_#Y.#m.#d_#H.#M.#s.log2021.03.09_05.35.35.log

Somehow, the variables cannot be resolved (_#Y.#m.#d_#H.#M.#s.log) , but then again is being resolved  (2021.03.09_05.35.35.log)

When I check the Rolling Log Option of the Data Step Batch Server, I can see:

 

Checking the sasv9_usermods.cfg I can see:

-logconfigloc /the/usual/path/to/servercontext/BatchServer/logconfig.apm.xml

In the file "logconfig.apm.xml" there is a block of appender code that:

   <!-- Batch File Appender, match the output log file name to the input -log statement -->
   <appender class="FileAppender" name="BatchFileAppender">
      <param name="Append" value="false"/>
      <param name="Unique" value="true"/>
      <param name="FileNamePattern" value="%S{App.Log}%d{yyyy.MM.dd_hh.mm.ss}.log"/>
      <layout>
         <param name="ConversionPattern" value="%m"/>
      </layout>
   </appender>

Is this the responsible part of the code, that leads to the fact, that logfiles of batch jobs are resolved to the green part of the example logfile name?

You suggested resetting the command, by doing the following

1. "Reset" the Rolling Log Option to "_" 
2. Redeploy the job in question
3. Check Command-Syntax of Job

I have done all three steps:

ad 1.) Option is set to "_"

ad 2.) Job was successfully redeployed

ad 3.) still the command looks like this:

/sas_install_p/config/Lev3/SASDI/BatchServer/sasbatch.sh -log /logs_p/Batch/Test_#Y.#m.#d_#H.#M.#s.log  -batch -noterminal -logparm "rollover=session"  -sysin /sas_p/lsf_jobs/Test.sas

Do I have to restart the Metadata-Server in order for the Rolling Log Option to take effect? 

[gwootton]

No, you shouldn't have to restart the Metadata Server. Redeploying should change the command line. There was a hot fix pointed out in a previous thread that might apply here:
https://communities.sas.com/t5/Administration-and-Deployment/How-to-change-the-default-log-file-loca...

There is also a reset button for the command line if it's been changed from the default that might need to be used.

I would also mention that if you are using logconfig.apm.xml, the parameter "_" is usually checked and reset automatically by the Service Architecture Framework processes, so it's odd you had to change that manually unless you have since disabled the framework without reverting the log configuration from logconfig.apm.xml to logconfig.xml.

[FK1]

Okay, this is really odd.
How can one "diable" the Service Architecture Framework processes?

[gwootton]

This is the documentation on disabling those processes.

Disabling the ACM and APM ETL Processes
https://go.documentation.sas.com/?cdcId=evcdc&cdcVersion=2.5_M1&docsetId=evug&docsetTarget=n1pt9o4oq...

[JackHamilton]

Where are you specifying the log file name? I think the # directives work only in the LOGPARM option, not in ALTLOG or LOG.
I ended up writing a script file to calculate the correct names.