topic Re: Wait until dataset is updated with today's data(date) in SAS Programming

Wait until dataset is updated with today's data(date)

Ranny — Wed, 09 Dec 2020 21:59:01 GMT

I have SAS dataset in below library -

Libname XYZ "/unix/mypath/project/202012";

SAS dataset - Project_dataset.sas7bdat

The above dataset has date_variable (mmddyy10.)

what I want is - If above dataset is available with today's date 12/09/2020 then move to the next step otherwise wait until today's data is updated.

How can we do this?

Thank you,

Renny

Re: Wait until dataset is updated with today's data(date)

SASKiwi — Wed, 09 Dec 2020 22:12:12 GMT

Am I correct in assuming that each day's dataset goes into a new folder? If so, IMO this is bad design as your LIBNAME has to point to a different folder each day.

One approach would be to use the FILEEXIST function (https://documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=9.4_3.5&docsetId=lefunctionsref&docsetTarget=n06xm8hwk0t0axn10gj16lfiri43.htm&locale=en)

to check if both the folder and dataset exist before doing any processing.

Re: Wait until dataset is updated with today's data(date)

ChrisNZ — Thu, 10 Dec 2020 00:44:39 GMT

If the daily data is appended, you can just test the last observation:

data _null_; 
  set SASHELP.CLASS nobs=NOBS; 
  set SASHELP.CLASS point=NOBS; 
  DATE=AGE;
  call symputx('dateok', ifn(DATE='17jan1960'd,1,0)); 
  stop;
run;

%put &=dateok;

Otherwise, you can use a where clause:

 data _null_; 
  call symputx('dateok', '0'); 
  set SASHELP.CLASS(where=(AGE=16)); 
  call symputx('dateok', '1'); 
  stop;
run;

%put &=dateok;

Re: Wait until dataset is updated with today's data(date)

Kurt_Bremser — Thu, 10 Dec 2020 08:14:04 GMT

See this macro code:

libname XYZ "/unix/mypath/project/202012";

%macro wait_till_ready;
%local today modate x;
%let today = %sysfunc(today());
proc sql noprint;
select datepart(modate) into :modate
from dictionary.tables
where libname = "XYZ" and memname = "PROJECT_DATASET"
;
quit;
%if &sqlobs = 0
%then %do;
  %put "Dataset does not exist!";
  %abort cancel;
%end;
%do %while (&modate ne &today);
  %let x = %sysfunc(sleep(60,1)); /* wait a minute */
  proc sql noprint;
  select datepart(modate) into :modate
  from dictionary.tables
  where libname = "XYZ" and memname = "PROJECT_DATASET"
  ;
  quit;
  %if %sysfunc(today()) gt &today
  %then %do;
    %put "No new data for this day!";
    %abort cancel;
  %end;
%end;
%mend;

The macro will check first if the dataset exists; if not, a message is issued and an error condition raised, and in a non-interactive session (batch), the session is terminated.

It retrieves the modification timestamp from DICTIONARY.TABLES, extracts the date, and compares that to the rundate of the job. If the dates don't match, it waits for a minute,

If the current date exceeds the rundate, a message is issued and the job/submit terminated abnormally.

Re: Wait until dataset is updated with today's data(date)

Ranny — Wed, 14 Jun 2023 21:43:02 GMT

SAS help class dataset - proc transpose

Re: Wait until dataset is updated with today's data(date)

ballardw — Thu, 15 Jun 2023 00:35:48 GMT

@Ranny wrote:

SAS help class dataset - proc transpose

????

And how does that attachment relate to your question from 2 years ago?