05-15-2017 11:42 AM
I have a datastep that routinely gets stuck at certain observations for reasons slightly beyond my control or ability. However a workaround for me is to interrupt the data step after sometime and work with whatever records I get. Any IDea how I can achieve that ?
Basically I want the data step to stop after say 5 minutes and give me whatever number of observations it curently holds.
05-15-2017 11:49 AM
You can't stop a data step based on minutes.
You can say that you will only work on the first X observations and then stop. Use the OBS= option.
But a far better solution would be to figure out why you are having this problem and prevent the problem, or fix it.
05-15-2017 12:04 PM
05-15-2017 12:22 PM
FIrst make sure you are taking advantage of the indexes that exist on the dictionary tables (virtual tables?) be avoiding using functions on LIBNAME or MEMNAME fields. Also use the PROC SQL and the DICTIONARY libref instead of using the views in the SASHELP.
Doing this should allow you to use WHERE conditions to limit the searching to the libraries and tables that you want.
** BAD ** ; where libname = upcase("&libref"); ** GOOD ** ; where libname = %upcase("&Lifbref") ;
Otherwise switch to use PROC CONTENTS instead.
05-16-2017 04:06 AM
I did start with using dictionary.tables and then proc contents and currently im using sashelp views for the reason - my SAS gui atleasts remains accessible and I can manually interrupt the data step from session management. with proc sql on dictionary.tables i had to completely terminate the session with no visible output.
05-15-2017 12:32 PM
Retain a datetime stamp at start running.
Compute end time by adding to 5 minutes or any other amount.
Check at every observation is end time reached and stop when done.
05-16-2017 04:04 AM
Need a timer that can run indepently of the Data step (trouble being the processing usually gets stuck at reading some observations and may stay there for upto 3 hours) . so checking at the beginning of each record does not help me. Im open to doing this by somehow embedding OS commands in it. But would like it to be part of overall code.
05-16-2017 04:14 AM
Would knowledge of the last observation executed or read will help to focus on the problematic one ?
You can define a filename.
For each observation save info to that file - rewriting it, one row only.
After canceling the job, you will have info of the last observation saved.
- read new obs
- open file rewrite mode
- write info
- close file
- deal with the obs
05-16-2017 10:32 AM