03-25-2016 06:19 AM
Since 9.4M3 (I think) there is a new entry field in the properties of the DATA Step Batch Server in tyhe Management Console. Below the Logs Directory we now have one for Output Directory.
I can imagine what this field can do (in fact have been hoping for this a long time) but I cannot see how it works out. The online docs and the in-app help do not mention the field. As far as I can tell it is not used when calling SAS. I am on RHEL7 Linux.
Many thanks in advance for you thoughts,
05-27-2016 06:08 AM
So I talked about this at the Data Management booth in the Quad and a few eyebrows were raised but noone had the answer. So they suggested to open a track with tech Support which I did. And lo and behold the answer came swift and caused a facepalm moment for me.
The Output Directory property of the Data Step Batch server works in concert with the "Send the output to a file" option of the Schedule Manager in SMC. If you right-click on the Schedule Manager plugin you get access to that option. So if that option is checked and you have a location specified for the batch server than the -print option is added to the SAS command that is composed when deploying the job. Easy as that. I did not find any reference to this option in the online or in-app help. When naming the .lst file the "Prefix the SAS log file with the flow name" option is also honoured. Just no timestamps or anything so output files will be overwritten.
So finally the riddle is solved. Thanks for all chiming in and I hope we all learned something.
03-25-2016 06:24 AM
this directory stands out for the default location of lst files generated by batch processes (proc print).
The .LST file is the output file for the program that has been executed. This file is created only if there is output for the program.
Before this, all the logs directory was filled with lst files, by default. Now it is possible to define a different location for those files, which can be useful as they can be used for controlling the batch processes and reporting.
I hope this can help a bit
03-25-2016 07:10 AM
Thanks for chiming in. That does make sense and it is (almost) exactly what I was hoping for. However, the expected does not happen. At least not for me. I have set it up as in the image but a job containing a proc print fails with
ERROR: Insufficient authorization to access /srv/SASConfig/Lev1/SASApp/Options_Libnames.lst.
So SAS tries to stick the .lst file in the root of the batch server config, ignoring the setting in the metadata. Also the command given to LSF to be execute does not contain the metadata setting. I would have expected a -print parameter.
I have taken care of .lst file locations many times, mostly by editing the sasbatch.sh script, but was hoping for things to become more automatic in 9.4M3. So I must be doing something wrong or missing something here. But I don't see it.
03-26-2016 08:50 AM
Thanks @LinusH. Yes I have. Many times.
The batch server is in fact not even a real server. It is a fancy way of defining the command line that's sent to the OS, in my case via LSF. The command is determined it two stages: first from metadata when deploying (eg from DI Studio) and later by adding the job to a flow. I have noticed that the latter tweaks the log line by adding the names of flow user to the logfile name. I have not added logconfigloc so no log4sas.
In all this I see no sign of adding or handling procedure output via the -print parameter, as the new server metadata property would suggest.
I hope for someone from the forum to shed some light in this. If not I will stalk the developer(s) at SGF ;-)
03-28-2016 06:00 AM
Jan. You are the question "what is an application". Please try to get one correct asap. It is very basic business process organisation.
I am having for you:
++ http://support.sas.com/publishing/pubcat/chaps/58172.pdf Tuning SAS Applications in the OS/390 and z/OS Environments. " What Is a SAS Application? The word application, as used in this book, is defined as one or more related SAS language programs that perform a specific function for an organization on a cyclic basis."
++ https://en.wikipedia.org/wiki/Application_software "An application program (app or application for short) is a computer program designed to perform a group of coordinated functions, tasks, or activities for the benefit of the user. "
Perhaps better to indicate it as https://en.wikipedia.org/wiki/Business_logic
- logic tier = code , the business sas programs (also map schedule informatie etc)
- data tier = data , the business data-sets (SAS-datasets Excel TD Oracle or whatever )
- presentation layer = results (lst or other like pdf/html/ods catalogs)
Very classic but not less true. It is te Foundation for ICT services when adding scheduling SLA's with technical performance.
After that you find the business goal is lost. All technical references are making the tools / utilities the leading subject (eg the SAS System) ignoring business requiements and business semantics. What are those business appplication in their world?
Examples of SAS applications can be: IH OB DM (just abbrevations for used business logic)
It is getting data in applying some business logic and getting results with logs in a scheduled way. (4 categories). Those are commonly build on the Develop/Test/Acceptance/Production stages (4 categories).
Now the question: "What is a SAS Appserver about.."
1/ Is it for the sake of an IT-department delivering the SAS-system as tool?
..... (won't proceed wiht this one)
2/ Is for fullfilling the business needs like OB IH DM?
..... this one makes the most sense for getting aligned with the business.
Yup some more work to do for IT departments. Can be automatized that is what it professionals can do.
Suppose you have those three businsess applications and some adhoc work how to organize storage locations?
You probably know this one: http://support.sas.com/kb/50/345.html The often mentioned bypass is hardcoding phyiscal names into business logic code. The result is frustrating the requirement of release management process.
You can solve it all by having four appservers (adhoc and three business logic) app-servers.
The destination of the logs/results/source repositories van be tailored as of each app-server. The batch server script can be aligned wiht that according to needed scheduling and security requiements.
03-29-2016 04:01 AM
Hi ja karman,
Regardless of the fact whether what you state is relevant or not I fail to see the relation of your reply to the question of the OP.
IMHO the OP indicates that the new option that is currently available for the batch server, that is to redirect output (.lst files) to a location of your own choosing, does not seem to live up to expectations.
Basically the OP would like to know if he is overlooking something causing it not to work for him.
I have seen the same option and also for me it is not working yet.
So if anyone has find the solution I am also curious to hear it. ;-)
03-29-2016 07:24 AM
03-29-2016 07:43 AM
Hi ja karman,
The fact that there may be more background within a project you are working on may very well be the case.
However your information is only taking this thread off-topic. Your technical and/or functional requirements may be different to what the feature provides but that is a topic for discussion inside your project. This thread is about a feature currently available.
IMHO the question as stated by the OP is simple:
There is seemingly a new feature in the SMC for the batch server and it is not working for him.
All the OP is interested in (and so am I) is how to get this feature working or, if misinterpreted, what is the actual meaning of the new feature.
03-29-2016 08:13 AM
03-30-2016 05:12 AM
Sorry @jakarman but your assumptious about what is being asked here are completey wrong. We are trying to get an answer to one single question: what is the usage of the Output Directory property of the Batch Server. Several things go wrong in your line of answering that result in total derailment of the thread that make me afraid I will not get an answer at all:
1) You assume a different question should be asked and answer the question you think should be asked. Your assumption is wrong and thus your answer.
2) Your answer is lengthy and its language confusing and inconcistent. You lost me at the second sentence.
3) You assume local knowledge from the workfloor ("OB DM IH ...") that we share to be the basis of my question. This assumption is also wrong. Otherwise approach me on that workfloor and not on the forum.
4) You refer to local knowledge that no one will be able to comprehend. Wrong again.
5) You target @Resa based on an incorrect assumption of what I am asking. You also incorrectly assume Resa has some batch thing that is not working. He doesn't, nor have I.
How can answering a simple question go wrong in so many ways?
Another simple question to you: will you please never answer any of my questions?
03-30-2016 07:13 AM
I also think it's being ignored. I tried it out with a SAS 9.4 M3 install on Linux deploying and scheduling a trivial single job flow. Looking at the command line within SAS Management Console, whilst the -log parameter is specified, there was no -print parameter for controlling the output file location. This was confirmed as absent by looking in the LSF logs too. This particular job then failed because it was trying to write a .lst file into the main SASApp dir where the user had no permissions (instead of the specified SASApp/BatchServer/Output dir where they would have).
I'm wondering if this is a) a defect or b) an option that is only used by some products/solutions (or planned for future products)?. Perhaps SAS Tech Support might be able to advise?
03-30-2016 07:28 AM
Thanks @PaulHomes for confirming my observations. And for putting the thread on track again ;-).
I am still pondering if I will report this to TS as it is not a clear bug. Also there is no immediate impact and we already have a long list of tracks. I am now consider addressing the issue at SGF in the Quad. And in that case we'll have plenty of opportunity to dicsuss the outcome.
Either way I will keep you posted.
05-10-2016 08:46 AM
In your last post you indicated:
I am now consider addressing the issue at SGF in the Quad. And in that case we'll have plenty of opportunity to dicsuss the outcome.
Out of curiosity (although I know what this did to the cat) ....
Did you find time to address this at SGF?
And if so, any news on this item?
Need further help from the community? Please ask a new question.