LSF Job Parameters

Reply
Contributor
Posts: 54

LSF Job Parameters

In LSF scheduler, it says hours, minutes and duration of event.  I do not understand what does 'duration of event' means ?  Can anyone explain please.

Thanks

-P

Trusted Advisor
Posts: 1,301

Re: LSF Job Parameters

You are referring to scheduling time events.

Time events are durations of time that are a set of dependency conditions consisting of:

1. a start date and time when the event becomes active

2. a duration in which the event remains active

Once current time falls withing the start time and duration, the time event becomes active and triggers the job execution

Contributor
Posts: 54

Re: LSF Job Parameters

Thank you !

My email notification is not working sometime when job fails despite under 'attributes' I've checked the 'email notification' check box in addition to 'flow ends' check box and 'flow end with errors' but not receiving emails all the times.  This is very annoying sometimes so your input would be very helpful.

Regards,

-P

Trusted Advisor
Posts: 1,301

Re: LSF Job Parameters

Are you sure the jobs are actually getting submitted.  If you are setting time event windows, it is possible that, without adequate duration for the event, it never gets submitted.

Contributor
Posts: 54

Re: LSF Job Parameters

Yes, they are submitted, I see the errors in the log but no notification emails. 

In addition, what would be the adequate event windows settings you prefer or normally do ?

Thanks

Trusted Advisor
Posts: 1,301

Re: LSF Job Parameters

It would be entirely situation based, there is no default.  I also no not typically schedule time events as it's not something I have a need for.  I would need to know more information about the flow you are attempting to build to give you any specific help.  I also almost never use the GUI interfaces for scheduling, so a lot of the descriptions on what you try are mostly lost on me.

Contributor
Posts: 54

Re: LSF Job Parameters

Our flows are very straight forward.  They need to run on a specific time on a daily weekly and monthly basis.  What would be standard approach based on your experience ? Thx.

Trusted Advisor
Posts: 1,301

Re: LSF Job Parameters

How many steps are in your flow?

What is their dependency structure?

What are the job level attributes are you using?  Job Resources?  Queue Resources?  Host Resources?

What queues are the jobs submitted to and what are the time event settings for them?

What hosts are the jobs submitted to and what are the time event settings for them?

Are your processes do any cross-communicating or messaging with LSF?

Are you using job arrays?  Queue Arrays? Clusters?


LSF is a very robust tool, regardless of how straight forward it may appear, there are many, many variables to consider.

Contributor
Posts: 54

Re: LSF Job Parameters

How many steps are in your flow? 2-5 jobs with any exit code

What is their dependency structure? no dependency

What are the job level attributes are you using?  Job Resources?  Queue Resources?  Host Resources?  None

What queues are the jobs submitted to and what are the time event settings for them? Time event settings are from 1 to 60.  Does this setting matters ?  Most of my jobs are set to '1'.

What hosts are the jobs submitted to and what are the time event settings for them? Windows SASApp Server.  The time event settings play any role in the job kick off.  My understanding is that the job will be kicked off as soon as it reaches the time assigned to the flow.  Do we need to provide more time for time event settings ?

Are your processes do any cross-communicating or messaging with LSF?  nope

Are you using job arrays?  Queue Arrays? Clusters? nope

Trusted Advisor
Posts: 1,301

Re: LSF Job Parameters

If there is no inner dependency structure, why are the jobs built in a flow?

Contributor
Posts: 54

Re: LSF Job Parameters

some jobs do.  Like previous ones needs to be completed successfully so the next one can kick off. very simple dependency.  In some cases, jobs are not starting at all.  This tool is behaving very differently all the times.  No consistency at all .....

Trusted Advisor
Posts: 1,301

Re: LSF Job Parameters

I'm sure the tool is only acting as erratically as it was told to. 

I have been using LSF for over 10 years, sometimes on clusters processing millions of jobs per month and flows consisting of thousands of jobs.  My experience has always been rock solid.

Contributor
Posts: 54

Re: LSF Job Parameters

I agree with you that it's a great tool but sometimes it gives hard time, not necessarily it's a tool issue but understanding one ....!

I see most of my jobs in LSF as 'job [abc.sas] in pending status', what does mean.  Please note that all the preceding jobs have been completed already but waiting in this mode for more than an hour is not understood.  Pl. advise !

Trusted Advisor
Posts: 1,301

Re: LSF Job Parameters

In your command prompt window, please type the following command:

bjobs -pl

This will give the job details and the pending reason.

The PEND state simple means that the job has been registered in LSF, but has not yet ran due to any of a number of reasons.  If you do not know how to interpret the output of the above command, paste it here.

Contributor
Posts: 54

Re: LSF Job Parameters

typed bjobs -pl on windows command prompt but got the message 'no pending jobs found' .....

Ask a Question
Discussion stats
  • 21 replies
  • 570 views
  • 0 likes
  • 5 in conversation