topic Re: output touch file (empty file) when processing is done in SAS Programming

output touch file (empty file) when processing is done

woo — Mon, 06 May 2019 13:23:03 GMT

Hello, i have to read db tables when

1) db table is fully loaded for a day (loading time is very in between specific time, lets say 1 am to 3 am) then i do some processing

2) at end of the process, i have to export 3 sas dataset into 3 different text file and once this export is done - i want to create 3 file (lets say simple empty text file without any data) which says okay, data files are out there and ready - so that other process (not SAS from now on) can read that empty file and further processing can happen - how to output empty file (touch file) once export is done,

Thanks.

%macro test;

data _null_;
set sashelp.stocks;
call symput ("nofobs", _n_);
run;

%put &nofobs;

%if &nofobs. >= 1 and Date = '1Aug2003'd %then
%do;

data test;
set sashelp.stocks;
run;

/*output text files*/
proc export data=test outfile='some share drive\datafile1.txt' dbms=tab replace;
putnames=no;
run;

proc export data=test2 outfile='some share drive\datafile2.txt' dbms=tab replace;
putnames=no;
run;

proc export data=test3 outfile='some share drive\datafile3.txt' dbms=tab replace;
putnames=no;
run;

/*once those three text file done, want to touch empty 3 file */
/*some logic */

filename mymail email "email"
subject="sas data ready alert ";

data _null_;
file mymail;
put "Hello team,";
put "SAS output is avilable, plese use data and process further";
run;

%end;

%else %do;

filename mymail email "email"
subject="oracle table alert";

data _null_;
file mymail;
put "Hello team,";
put "Table doesn't have enough records, sas is not going to process the table";
run;

%end;

%mend;

%test;

Re: output touch file (empty file) when processing is done

ChrisNZ — Tue, 07 May 2019 04:24:12 GMT

1. Like this?

data _null_;
   file "&path\name1.txt";
   put ' ';
   file "&path\name2.txt";
   put ' ';
   file "&path\name3.txt";
   put ' ';
run;

2. This test will never be true:

%if Date = '1Aug2003'd

3. Correct the many spelling errors.

Re: output touch file (empty file) when processing is done

ScottBass — Tue, 07 May 2019 05:31:19 GMT

data _null_;
set sashelp.stocks;
call symput ("nofobs", _n_);
run;

What if sashelp.stocks is 100M records? You're going to call symput() 100M times, overwriting nofobs 100M times, only to save the last result. That is, ahem, "inefficient".

Number of observations != empty/not empty.

Have a look at

https://github.com/scottbass/SAS/blob/master/Macro/nobs.sas or

https://github.com/scottbass/SAS/blob/master/Macro/check_if_empty.sas

/*once those three text file done, want to touch empty 3 file */
/*some logic */

Why do you need to create three marker files? If this is just to control downstream processing, then isn't one marker file enough?

Are the marker files meant to be the same name as your exported files? If so, of course create them before your export, otherwise they will overwrite your exported SAS files.

FWIW, I usually implemented the logic:

If marker file exists then ETL is running, don't run downstream code.

If marker file does not exist then upstream ETL was successful, run downstream code.

I then create the "ETL is running" marker file at the beginning of the ETL, and delete the marker file when the success condition is met. The downstream ETL loops while the marker file exists.

Then if I come in the morning and the marker file exists I know where the error occurred. Of course, the email tells me that too :). I then fix the issue, manually delete the marker file, then re-run the downstream ETL.

Finally, you might be able to steal some of the logic in here:

https://github.com/scottbass/SAS/blob/master/Macro/lock.sas

https://github.com/scottbass/SAS/blob/master/Macro/marker.sas

It's not exactly what you're looking for but may give you some ideas...

Re: output touch file (empty file) when processing is done

woo — Tue, 07 May 2019 13:49:00 GMT

Thanks for your input,

i changed logic and came up with this, can i make this better?

%macro test;

%let todaysdate=%sysfunc(today(), date8.);
%put &todaysdate;

/*have temp dataset which has "Date" variable like 05May2019*/

libname mylib "/path";

data _null_;
set mylib.test1 nobs=n;
call symput ("nobs", n);
call symput ("tempdate",put(Date, date8.));
run;

%put &nobs;

%if &nobs. >= 1 and &todaysdate.=&tempdate. %then

%do;

data test;
set sashelp.stocks;
run;

/*output results in text files*/

proc export data=test outfile='/sharedrive/datafile1.txt' dbms=tab replace;
putnames=no;
run;

/*touch file after export is done */

%let testfile1="/sharedrive/datafile1.txt";

%if %sysfunc(fileexist(&testfile1)) %then do;

data _null_;
infile "cd /sharedrive; touch datafile1_ready.txt;" pipe;
run;

%end;

/*same logic, export + touchfile, for file2 and file3*/

%end;

/*same logic for other two text files*/

%else

%do;

filename mymail email "email"
subject="sas data ready alert ";

data _null_;
file mymail;
put "Hello team,";
put "input file doesn't have enough data";
run;

%end;

%mend;

%test;

Re: output touch file (empty file) when processing is done

Tom — Tue, 07 May 2019 19:29:10 GMT

What types of values would DATE8. format generate? DATE needs 9 characters, 2 for day, 3 for month and 4 for year.

Use DATE9.

Don't use the old style CALL SYMPUT(). Use the newer CALL SYMPUTX() instead.

if the input dataset is empty the step will stop at the SET statement and never get to the CALL SYMPUTX() statement. So write the value of NOBS macro variable BEFORE the SET statement. And make sure to define TEMPDATE macro variable in case there are no observations in the dataset. No need to read the whole dataset.

%let tempdate=UNKNOWN;
data _null_;
  call symputX("nobs", n);
  set mylib.test1 nobs=n;
  call symputX("tempdate",put(Date, date9.));
  stop;
run;