I have problem when I am executing the STP to run on LSF using the command “bsub”. It looks like some credential are changing for myuser. (and becomes storage issue apparently).
I execut bsub and the name of the store process and if the work tables in that storeed process are small I get log file and completet program. If the work tables are bigger, the program stops without writhing in the log file.
If I put some inputs that I have found on the IBM web site I get ERROR in my log file. here is what I write in the cod:
bsub -C 500MB -M 1G -S 1TB -F 1M -R "rusage[mem=512MB:swp=1GB:tmp=1TB]"
I have tested and played with the vaules here but I didn't suceed.
I need help using this command "bsub", so it will not change the setings for my user on the server.
Work is a SAS option and you should set it at SAS Configuration level. Using LSF properties to control resources can become fairly complex at times. You have mentioned that when the work tables are small, STP is working fine so it could be that you are running out of space on the assigned work library. You should raise this concern to your site administrator to help you allocate a different work location.
If you have access to login to the server you can set this library dynamically while invoking a SAS session from command line by providing the -work '<your-file-system-path>' option.
Ex: ..../SASFoundation/9.4/sas -sysin MySASCode.sas -work /path-to-my-filesystem
Hope this helps. Also, it would be great if you can post some errors you are facing to help us understand your issue better.
This user is updating bilion rows evry night and it newer been problem with the work space.
If I use only bsub , tho job just stops without writing in the log.
If I use bsub -C 500MB -M 1G -S 1TB -F 1M -R "rusage[mem=512MB:swp=1GB:tmp=1TB]" Than I get the ERROR in the log:
"ERROR: Insufficient space in file WORK.QUERY_FOR_.DATA"
I have plaeyed with all of this -C -W -M -S -F and If I set too big the job is pendinging and It newer starts.
The sheduled DI job calls a script like this:
'bsub "sh /opt/....../send_to_lsf.sh "SOME PARAMETARS WHERE THE STP's are" "';
and the script is doing following
/opt/.../batch.sh -log /opt/..../stp_#Y.#m.#d_#H.#M.#s.log -batch -noterminal -logparm "rollover=session" -sysin /opt/.../executestp.sas -sysparm "$1"
yes is I run the same STP with the same user on the same mashin with proc STP comand works fine.
the idea using bsub is to Control the STPs That they don't run in the same time sing the Flow Manager have queue system
If I use only -F for exaple and if the work table is too big, than it fails if I use -F 1000 (and no info in the log), If I user -F 8000 (I get ERROR in the log on the first wrok table in the cod), If I incerease this a lot than the job doesn't starts it is pending in the queue.
bsub -F 1000 /opt/.../sbatch.sh -log /opt/..../STP.log -batch -noterminal -logparm 'rollover=session' -sysin /opt/..../execute_user_stp_start.sas -sysparm '/Shared Data/......./NAME_OF_STP'
bsub -F 8000 /opt/.../sbatch.sh -log /opt/..../STP.log -batch -noterminal -logparm 'rollover=session' -sysin /opt/..../execute_user_stp_start.sas -sysparm '/Shared Data/......./NAME_OF_STP'
My problem is That I don't know how the credentiasl change with bsub.
to answer to your question regarding LSF_UNIT_FOR_LIMITS I havent done any Changes.
My suggestion is to use the SAS Grid Manager client utility to submit batch jobs to Grid. You can find more information about it here -> https://go.documentation.sas.com/?docsetId=gridref&docsetTarget=p06nu8c3qifl8vn15ya1ghu97lku.htm&doc...
I haven't tried invoking sasbatch.sh directly using bsub, however for anything other than sas programs bsub would fail. As the LSF license that comes with SAS is OEM and NOT full version. It allows you to submit .sas files only.
There is a good answer by Patrick explaining this here -> https://communities.sas.com/t5/SAS-Data-Management/Scheduling-Unix-Script-in-LSF/td-p/103904
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.