Architecting, installing and maintaining your SAS environment

Output Directory in Batch Server properties

Accepted Solution Solved
Reply
Super Contributor
Posts: 408
Accepted Solution

Output Directory in Batch Server properties

Hi,

 

Since 9.4M3 (I think) there is a new entry field in the properties of the DATA Step Batch Server in tyhe Management Console. Below the Logs Directory we now have one for Output Directory.

 

2016-03-25 11_14_46-Standaard vDesktop Ont 2015.1.png

 

I can imagine what this field can do (in fact have been hoping for this a long time) but I cannot see how it works out. The online docs and the in-app help do not mention the field. As far as I can tell it is not used when calling SAS. I am on RHEL7 Linux.

 

Many thanks in advance for you thoughts,

- Jan.


Accepted Solutions
Solution
‎05-28-2016 08:14 AM
Super Contributor
Posts: 408

Re: Output Directory in Batch Server properties

So I talked about this at the Data Management booth in the Quad and a few eyebrows were raised but noone had the answer. So they suggested to open a track with tech Support which I did. And lo and behold the answer came swift and caused a facepalm moment for me.

 

The Output Directory property of the Data Step Batch server works in concert with the "Send the output to a file" option of the Schedule Manager in SMC. If you right-click on the Schedule Manager plugin you get access to that option. So if that option is checked and you have a location specified for the batch server than the -print option is added to the SAS command that is composed when deploying the job. Easy as that. I did not find any reference to this option in the online or in-app help. When naming the .lst file the "Prefix the SAS log file with the flow name" option is also honoured. Just no timestamps or anything so output files will be overwritten.

 

So finally the riddle is solved. Thanks for all chiming in and I hope we all learned something.

 

Regards,

- Jan.

View solution in original post


All Replies
Trusted Advisor
Posts: 1,141

Re: Output Directory in Batch Server properties

Hi @jklaverstijn,

 

this directory stands out for the default location of lst files generated by batch processes (proc print).

 

The .LST file is the output file for the program that has been executed. This file is created only if there is output for the program.

 

Before this, all the logs directory was filled with lst files, by default. Now it is possible to define a different location for those files, which can be useful as they can be used for controlling the batch processes and reporting.

 

I hope this can help a bit Smiley Happy

Super Contributor
Posts: 408

Re: Output Directory in Batch Server properties

Hi @JuanS_OCS,

 

Thanks for chiming in. That does make sense and it is (almost) exactly what I was hoping for. However, the expected does not happen. At least not for me. I have set it up as in the image but a job containing a proc print fails with

 

ERROR: Insufficient authorization to access /srv/SASConfig/Lev1/SASApp/Options_Libnames.lst.

 

 

So SAS tries to stick the .lst file in the root of the batch server config, ignoring the setting in the metadata. Also the command given to LSF to be execute does not contain the metadata setting. I would have expected a -print parameter.

 

I have taken care of .lst file locations many times, mostly by editing the sasbatch.sh script, but was hoping for things to become more automatic in 9.4M3. So I must be doing something wrong or missing something here. But I don't see it. 

 

- Jan.

Super User
Posts: 5,256

Re: Output Directory in Batch Server properties

Have you redeployed your jobs? Since the batch server doesn't call the meta data server during execution.
Data never sleeps
Super Contributor
Posts: 408

Re: Output Directory in Batch Server properties

Thanks @LinusH. Yes I have. Many times.

 

The batch server is in fact not even a real server. It is a fancy way of defining the command line that's sent to the OS, in my case via LSF. The command is determined it two stages: first from metadata when deploying (eg from DI Studio) and later by adding the job to a flow. I have noticed that the latter tweaks the log line by adding the names of flow user to the logfile name. I have not added logconfigloc so no log4sas.

 

In all this I see no sign of adding or handling procedure output via the -print parameter, as the new server metadata property would suggest.

 

I hope for someone from the forum to shed some light in this. If not I will stalk the developer(s) at SGF ;-)

 

- Jan.

Valued Guide
Posts: 3,208

Re: Output Directory in Batch Server properties

Jan. You are the question "what is an application".  Please try to get one correct asap. It is very basic business process organisation.  
I am having for you:

++ http://support.sas.com/publishing/pubcat/chaps/58172.pdf   Tuning SAS Applications in the OS/390 and z/OS Environments.   "  What Is a SAS Application?   The word application, as used in this book, is defined as one or more related SAS language programs that perform a specific function for an organization on a cyclic basis."
++ https://en.wikipedia.org/wiki/Application_software "An application program (app or application for short) is a computer program designed to perform a group of coordinated functions, tasks, or activities for the benefit of the user. "

 

Perhaps better to indicate it as https://en.wikipedia.org/wiki/Business_logic 

- logic tier  = code , the business sas programs (also map schedule informatie etc)

- data tier  = data  , the business data-sets (SAS-datasets Excel TD Oracle or whatever ) 

- presentation layer = results (lst or other like pdf/html/ods catalogs)

Very classic but not less true. It is te Foundation for ICT services when adding scheduling SLA's with technical performance.

 

After that you find the business goal is lost. All technical references are making the tools / utilities the leading subject (eg the SAS System) ignoring business requiements and business semantics. What are those business appplication in their world?
Examples of SAS applications can be: IH OB DM     (just abbrevations for used business logic)  

It is getting data in applying some business logic and getting results with logs in a scheduled way. (4 categories). Those are commonly build on the Develop/Test/Acceptance/Production stages (4 categories). 

Now the question: "What is a SAS Appserver about.."

 1/ Is it for the sake of an IT-department delivering the SAS-system as tool? 

     .....  (won't proceed wiht this one) 

 2/ Is for fullfilling the business needs like OB IH DM? 

     .....  this one makes the most sense for getting aligned with the business.
   Yup some more work to do for IT departments. Can be automatized that is what it professionals can do. 

Suppose you have those three businsess applications and some adhoc work how to organize storage locations?

You probably know this one: http://support.sas.com/kb/50/345.html The often mentioned bypass is hardcoding phyiscal names into business logic code. The result is frustrating the requirement of release management process.
You can solve it all by having four appservers (adhoc and three business logic) app-servers.

The destination of the logs/results/source repositories van be tailored as of each app-server. The batch server script can be aligned wiht that according to needed scheduling and security requiements.

---->-- ja karman --<-----
Frequent Contributor
Posts: 84

Re: Output Directory in Batch Server properties

Hi ja karman,

Regardless of the fact whether what you state is relevant or not I fail to see the relation of your reply to the question of the OP.

 

IMHO the OP indicates that the new option that is currently available for the batch server, that is to redirect output (.lst files) to a location of your own choosing, does not seem to live up to expectations.

Basically the OP would like to know if he is overlooking something causing it not to work for him.

 

I have seen the same option and also for me it is not working yet.

So if anyone has find the solution I am also curious to hear it.   ;-)

 

Valued Guide
Posts: 3,208

Re: Output Directory in Batch Server properties

Resa, there is some more background known than you can see following this thread question.
Those business applications OB DM IH are real ones aside of that adhoc to come a live in the infrastructure Jan is helping to deliver.
That one of the current directory ws is a real experienced one in that environment (temp files). Adding an option in the batch server for lst redirection is a technical one a previous described. I am answering the functional requirements. There is difference between technical working and functionality within business.
As weird it is. Do not use spaces in namings. Unix and Windows are allowing that, but the scripting by SAS fails.
---->-- ja karman --<-----
Frequent Contributor
Posts: 84

Re: Output Directory in Batch Server properties

Hi ja karman,

The fact that there may be more background within a project you are working on may very well be the case.

However your information is only taking this thread off-topic. Your technical and/or functional requirements may be different to what the feature provides but that is a topic for discussion inside your project. This thread is about a feature currently available.

 

IMHO the question as stated by the OP is simple:

There is seemingly a new feature in the SMC for the batch server and it is not working for him.

All the OP is interested in (and so am I) is how to get this feature working or, if misinterpreted, what is the actual meaning of the new feature.

 

 

Valued Guide
Posts: 3,208

Re: Output Directory in Batch Server properties

Resa, did you verify not having spaces in the namings somewhere? This is a problem with the batch server script still existing. It is caused by the logging settings.
The lst file also did cause issues as you must have full write access to the current directory.
Both issues are solved in our project with Jan. The lst is a nasty one. You have to have write access to the current map even if you do not create one. They are hardly used arround here and when it are functional logging.

I think when you take care setting the current dir as stated by that other note your batch script will work.
---->-- ja karman --<-----
Super Contributor
Posts: 408

Re: Output Directory in Batch Server properties

Sorry @jakarman but your assumptious about what is being asked here are completey wrong. We are trying to get an answer to one single question: what is the usage of the Output Directory property of the Batch Server. Several things go wrong in your line of answering that result in total derailment of the thread that make me afraid I will not get an answer at all:

 

1) You assume a different question should be asked and answer the question you think should be asked. Your assumption is wrong and thus your answer.
2) Your answer is lengthy and its language confusing and inconcistent. You lost me at the second sentence.
3) You assume local knowledge from the workfloor ("OB DM IH ...") that we share to be the basis of my question. This assumption is also wrong. Otherwise approach me on that workfloor and not on the forum.
4) You refer to local knowledge that no one will be able to comprehend. Wrong again.
5) You target @Resa based on an incorrect assumption of what I am asking. You also incorrectly assume Resa has some batch thing that is not working. He doesn't, nor have I.

 

How can answering a simple question go wrong in so many ways?

 

Another simple question to you: will you please never answer any of my questions?

 

Regards, Jan.

PROC Star
Posts: 392

Re: Output Directory in Batch Server properties

Hi Jan,

 

I also think it's being ignored. I tried it out with a SAS 9.4 M3 install on Linux deploying and scheduling a trivial single job flow. Looking at the command line within SAS Management Console, whilst the -log parameter is specified, there was no -print parameter for controlling the output file location. This was confirmed as absent by looking in the LSF logs too. This particular job then failed because it was trying to write a .lst file into the main SASApp dir where the user had no permissions (instead of the specified SASApp/BatchServer/Output dir where they would have).

 

I'm wondering if this is a) a defect or b) an option that is only used by some products/solutions (or planned for future products)?. Perhaps SAS Tech Support might be able to advise?

 

Cheers

Paul

Super Contributor
Posts: 408

Re: Output Directory in Batch Server properties

Thanks @PaulHomes for confirming my observations. And for putting the thread on track again ;-).

 

I am still pondering if I will report this to TS as it is not a clear bug. Also there is no immediate impact and we already have a long list of tracks. I am now consider addressing the issue at SGF in the Quad. And in that case we'll have plenty of opportunity to dicsuss the outcome.

 

Either way I will keep you posted.

 

Cheers Jan

Frequent Contributor
Posts: 84

Re: Output Directory in Batch Server properties

Hi Jan,

In your last post you indicated:

 

jklaverstijn wrote:

I am now consider addressing the issue at SGF in the Quad. And in that case we'll have plenty of opportunity to dicsuss the outcome.

Out of curiosity (although I know what this did to the cat) ....

Did you find time to address this at SGF?

And if so, any news on this item?

Super Contributor
Posts: 408

Re: Output Directory in Batch Server properties

Hi Resa,

Just got back from SGF and a subsequent vacation in SW USA. Still buzzing from the jetlag and all the impressions.

I talked to several guys in the data management booth and they were all stumped. General advise was to raise the issue with tech support, which I will do asap.

Cheers Jan.
☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 21 replies
  • 1321 views
  • 21 likes
  • 6 in conversation