Unix SAS Batch

[JJP1]

Hi,

Iam trying to resolve the error saying ""A lock is not available for xxx.name.data"

so would you please help on how can i resolve where i need to see whether lock is available on dataset or not,if lock available wait for locked dataset for few seconds and proceed with batch run.

The above issue i guess can be resolved through FILLEOCKWAIT option 
but i need to apply logic to write code on SAS batch programs that are running as part of UNIX BATCH please as these datasets are not avaialable through SAS applications.
please help

[SASKiwi]

FILELOCKWAIT will do what you require - pause the update of a SAS dataset for a few seconds until a lock has cleared. You don't have to code any extra logic.

[JJP1]

Hi @SASKiwi ,

I do not want to use FILELOCKWAIT option,so only i am contacting any other option please

[andreas_lds]

---

@JJP1 wrote:

Hi @SASKiwi ,

I do not want to use FILELOCKWAIT option,so only i am contacting any other option please

---

Why not?

[SASKiwi]

@JJP1 - Any other option will take coding, testing and support effort and may not work as well. I echo @andreas_lds 's why not.

[JJP1]

Thanks @SASKiwi ,

Any other option will take coding, testing and support effort and may not work as well.

Thanks SASKiwi,i am trying to learn is there any way to handle this case where SAS batch jobs are running under UNIX environment please.sorry to annoy.

[Patrick]

@JJP1 

You can use FILELOCKWAIT as part of a Libname statement. This doesn't require admin privileges.

Using FILELOCKWAIT is the best and easiest solution - by far!

[JJP1]

HI All,

As part of monthly job i am copying weekly dataset say "weekdataset" to monthly dataset "monthdataset" on every 1st of month.
there is also one more weekly job "selectname" which is using weekly dataset "weekdataset" at same time(it might or might not happen same time always but this issue  has occured only once)due to which i got lock error ""A lock is not available for vv.weekdataset.DATA"
in weekly job "selectname",as this dataset is being used in monthly job to copy over.

if we user FILE LOCK WAIT option blindly it will see the weekly dataset and run the job "selectname" with old data

Actually the above issue has occured only once not always and it is more of timing issue,it may occur and may not occur please.
so as part of this I am trying to fix permanently.please suggest or correct me.

[DWilson]

---

@JJP1 wrote:

HI All,

As part of monthly job i am copying weekly dataset say "weekdataset" to monthly dataset "monthdataset" on every 1st of month.
there is also one more weekly job "selectname" which is using weekly dataset "weekdataset" at same time(it might or might not happen same time always but this issue  has occured only once)due to which i got lock error ""A lock is not available for vv.weekdataset.DATA"
in weekly job "selectname",as this dataset is being used in monthly job to copy over.

if we user FILE LOCK WAIT option blindly it will see the weekly dataset and run the job "selectname" with old data

Actually the above issue has occured only once not always and it is more of timing issue,it may occur and may not occur please.
so as part of this I am trying to fix permanently.please suggest or correct me.

---

How will selectname use old data? Both applications are accessing a dataset named "weekdataset" and neither job, as you describe them, is updating "weekdataset." If you use FILE LOCK WAIT then how will that make selectname use a different data set than "weekdataset"? That makes no sense.

[JJP1]

not different dataset,same dataset weekdataset" please.you understood please

[DWilson]

---

@JJP1 wrote:
not different dataset,same dataset weekdataset" please.you understood please

---

But you said the process would read "old" data. If it's not reading old data, then what's the issue? It waits until the file is unlocked right?

[JJP1]

ok what if it did not wait for unlocked file to be available and run the job with previous weekly available data.and what if the weekly dataset is not updated with latest records and yet to run may be just seconds.actually it is more of timing issue,it may occur and may not occure. sorry to annoy but kindly help do we have any other option or you mean that there is no other way except "FILELOCKWAIT" please.

[DWilson]

---

@JJP1 wrote:
ok what if it did not wait for unlocked file to be available and run the job with previous weekly available data.and what if the weekly dataset is not updated with latest records and yet to run may be just seconds.actually it is more of timing issue,it may occur and may not occure. sorry to annoy but kindly help do we have any other option or you mean that there is no other way except "FILELOCKWAIT" please.

---

This doesn't seem to be solely a file lock issue. It seems that you are saying that your data are updated and you want your process to not run unless the data have been updated. That is not a file lock issue. You'll have to institute some sort of process to check the update status of the data. How you do that depends on your circumstances. One way would be to check the file time stamp to see if the file was updated recently; "recently" depends on how often your data are updated. If it's once a day then you can check to see if the file was updated the day your program runs; if it wasn't then delay your program. You'd have to put some sort of while loop in your SAS code to delay execution until the timestamp changes. Alternatively, use batch scripting to do the checking and once the file has been updated submit your SAS code. Your SAS code would still need the code to handle file lock issues.

[Patrick]

@JJP1 

So what you're dealing with are uncontrolled job dependencies. The job reading the data could eventually even execute before the data gets loaded and there wouldn't be any file locking but you still would read old data.

You could use a scheduler to execute your jobs and define the dependencies there. This way the job reading the data would never execute before the data has been loaded.

From what I understand your weekly data table gets fully replaced once a week. 

You could have a date/datetime column in your weekly table which has the date of your data. The job reading the data then checks if the date is already from the latest load (by comparing the date in the table with the current system date). In a do loop read one row from the table and check the date. If the date is not yet o.k. then issue a call sleep() and then continue looping, if the date is o.k. then leave the loop and continue processing all the data.

There are also other options like having a control table where your load job updates the date of the latest load - and your read job using this table to check if there is already anything to process.

[ScottBass]

---

@JJP1 wrote:
ok what if it did not wait for unlocked file to be available and run the job with previous weekly available data.and what if the weekly dataset is not updated with latest records and yet to run may be just seconds.actually it is more of timing issue,it may occur and may not occure. sorry to annoy but kindly help do we have any other option or you mean that there is no other way except "FILELOCKWAIT" please.

---

I'm not sure why FILELOCKWAIT doesn't meet your needs?  But since you insist it does not...

I agree with @Patrick that a robust scheduler which handles job dependencies would help here.  Otherwise, ways that I have handled this:

1) A control table, where you insert a row when jobname FOO starts, setting StartDatetime and NULL EndDatetime.  Downstream jobs that have a dependency on FOO select the row where jobname=FOO and StartDatetime = MAX(StartDatetime) (i.e. the latest row for jobname FOO).  If EndDatetime = NULL then pause your current job using the SLEEP function.  Loop until EndDatetime is not NULL or a configurable timeout occurs.  If EndDatetime is not NULL, check if it is within a desired timeframe, i.e. it completed within the last 24 hours rather than 3 weeks ago.  PLUS you need to contend with potential locking on the control table itself (although the locking would be very brief) if you submit many jobs at once.  Or license SAS/Share.  Or use an RBDMS table for your control table.

2) Alternatively, create a marker file when jobname FOO starts.  Check for the existence of the marker file in downstream code.  Loop if the file exists.  Delete the file when jobname FOO ends.  Obviously this is simpler to code but you lose the metrics captured via option #1 above.

All of this takes coding in all your jobs.  And of of this "stuff" is handled by a full-featured scheduler.

https://github.com/scottbass/SAS/blob/master/Macro/lock.sas may give you some further design ideas.  It was written for Windows so would need to be heavily tested for Unix.  And if you're new to SAS and Macro it may be gibberish to you.  But it may give you some ideas you can run with.