BookmarkSubscribeRSS Feed
blooneel
Calcite | Level 5

Hi Guys,

 

I have a datastep that routinely gets stuck at certain observations for reasons slightly beyond my control or ability. However a workaround for me is to interrupt the data step after sometime and work with whatever records I get. Any IDea how I can achieve that ?

 

Basically I want the data step to stop after say 5 minutes and give me whatever number of observations it curently holds.

10 REPLIES 10
PaigeMiller
Diamond | Level 26

You can't stop a data step based on minutes.

 

You can say that you will only work on the first X observations and then stop. Use the OBS= option.

 

But a far better solution would be to figure out why you are having this problem and prevent the problem, or fix it.

--
Paige Miller
blooneel
Calcite | Level 5
I have tried that . But I want to get as many OBS as possible. As to your other point im pulling observations from dictionary.tables and the code works absolutely fine on one of the servers and gets randomly stuck on the other one with no errors. There might be a solution but its well beyond my knowledge
Tom
Super User Tom
Super User

FIrst make sure you are taking advantage of the indexes that exist on the dictionary tables (virtual tables?) be avoiding using functions on LIBNAME or MEMNAME fields.  Also use the PROC SQL and the DICTIONARY libref instead of using the views in the SASHELP.

 

Doing this should allow you to use WHERE conditions to limit the searching to the libraries and tables that you want.

** BAD ** ;
where libname = upcase("&libref");

** GOOD ** ;
where libname = %upcase("&Lifbref") ;

Otherwise switch to use PROC CONTENTS instead.

 

blooneel
Calcite | Level 5

I did start with using dictionary.tables and then proc contents and currently im using sashelp views for the reason - my SAS gui atleasts remains accessible and I can manually interrupt the data step from session management. with proc sql on dictionary.tables i had to completely terminate the session with no visible output.

Shmuel
Garnet | Level 18

Retain a datetime stamp at start running.

Compute end time by adding to 5 minutes or any other amount.

Check at every observation is end time reached and stop when done.

blooneel
Calcite | Level 5

Need a timer that can run indepently of the Data step (trouble being the processing usually gets stuck at reading some observations and may stay there for upto 3 hours) . so checking at the beginning of each record does not help me. Im open to doing this by somehow embedding OS commands in it. But would like it to be part of overall code.

Shmuel
Garnet | Level 18

Would knowledge of the last observation executed or read will help to focus on the problematic one ?

 

You can define a filename.

For each observation save info to that file - rewriting it, one row only.

After canceling the job, you will have info of the last observation saved.

 

try:

   - read new obs

   - open file rewrite mode

   - write info

   - close file

   - deal with the obs

Peter_C
Rhodochrosite | Level 12
You are at risk of investigating irrelevant libraries.
The list of these librefs might vary vetween servers.
The impact on performance becomes greatest if for example you have one libref pointing into an excel workbook. The collector of metadata for dictionary.tables will investigate how many rows / columns are in each sheet and range name.
If a libref is sufficiently complete to open a connection to a database through SAS/ACCESS it might be trying to count the number of columns in a VIEW.
It might be sufficient to use dictionary.members which populates much more quickly - having fewer columns that risk far less investigation of the platform.

blooneel
Calcite | Level 5

Hey ,

 

Dictionary.Members was working quickly. Anyway I can get Encryption information from that ?

Peter_C
Rhodochrosite | Level 12
no

but a join with other metadata can be optimised on libname and memname

##- Please type your reply above this line. Simple formatting, no
attachments. -##

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 10 replies
  • 1003 views
  • 0 likes
  • 5 in conversation