Stop a datastep after certain time

Reply
Occasional Contributor
Posts: 9

Stop a datastep after certain time

Hi Guys,

 

I have a datastep that routinely gets stuck at certain observations for reasons slightly beyond my control or ability. However a workaround for me is to interrupt the data step after sometime and work with whatever records I get. Any IDea how I can achieve that ?

 

Basically I want the data step to stop after say 5 minutes and give me whatever number of observations it curently holds.

Trusted Advisor
Posts: 1,496

Re: Stop a datastep after certain time

You can't stop a data step based on minutes.

 

You can say that you will only work on the first X observations and then stop. Use the OBS= option.

 

But a far better solution would be to figure out why you are having this problem and prevent the problem, or fix it.

Occasional Contributor
Posts: 9

Re: Stop a datastep after certain time

I have tried that . But I want to get as many OBS as possible. As to your other point im pulling observations from dictionary.tables and the code works absolutely fine on one of the servers and gets randomly stuck on the other one with no errors. There might be a solution but its well beyond my knowledge
Super User
Super User
Posts: 6,318

Re: Stop a datastep after certain time

FIrst make sure you are taking advantage of the indexes that exist on the dictionary tables (virtual tables?) be avoiding using functions on LIBNAME or MEMNAME fields.  Also use the PROC SQL and the DICTIONARY libref instead of using the views in the SASHELP.

 

Doing this should allow you to use WHERE conditions to limit the searching to the libraries and tables that you want.

** BAD ** ;
where libname = upcase("&libref");

** GOOD ** ;
where libname = %upcase("&Lifbref") ;

Otherwise switch to use PROC CONTENTS instead.

 

Occasional Contributor
Posts: 9

Re: Stop a datastep after certain time

I did start with using dictionary.tables and then proc contents and currently im using sashelp views for the reason - my SAS gui atleasts remains accessible and I can manually interrupt the data step from session management. with proc sql on dictionary.tables i had to completely terminate the session with no visible output.

Super User
Posts: 1,228

Re: Stop a datastep after certain time

Retain a datetime stamp at start running.

Compute end time by adding to 5 minutes or any other amount.

Check at every observation is end time reached and stop when done.

Occasional Contributor
Posts: 9

Re: Stop a datastep after certain time

Need a timer that can run indepently of the Data step (trouble being the processing usually gets stuck at reading some observations and may stay there for upto 3 hours) . so checking at the beginning of each record does not help me. Im open to doing this by somehow embedding OS commands in it. But would like it to be part of overall code.

Super User
Posts: 1,228

Re: Stop a datastep after certain time

Would knowledge of the last observation executed or read will help to focus on the problematic one ?

 

You can define a filename.

For each observation save info to that file - rewriting it, one row only.

After canceling the job, you will have info of the last observation saved.

 

try:

   - read new obs

   - open file rewrite mode

   - write info

   - close file

   - deal with the obs

Valued Guide
Posts: 2,174

Re: Stop a datastep after certain time

You are at risk of investigating irrelevant libraries.
The list of these librefs might vary vetween servers.
The impact on performance becomes greatest if for example you have one libref pointing into an excel workbook. The collector of metadata for dictionary.tables will investigate how many rows / columns are in each sheet and range name.
If a libref is sufficiently complete to open a connection to a database through SAS/ACCESS it might be trying to count the number of columns in a VIEW.
It might be sufficient to use dictionary.members which populates much more quickly - having fewer columns that risk far less investigation of the platform.

Occasional Contributor
Posts: 9

Re: Stop a datastep after certain time

Hey ,

 

Dictionary.Members was working quickly. Anyway I can get Encryption information from that ?

Valued Guide
Posts: 2,174

Re: Stop a datastep after certain time

no

but a join with other metadata can be optimised on libname and memname

##- Please type your reply above this line. Simple formatting, no
attachments. -##
Ask a Question
Discussion stats
  • 10 replies
  • 141 views
  • 0 likes
  • 5 in conversation