BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
JJP1
Pyrite | Level 9

Hi Team
There is a user written code written which is sending 2 csv files as an attachment and sending e-mails to few people.
if there is no data it will say message like "no data" in e-mail with out attachment.

The requiremnet here is i need to implement the above scenario by making use of SAS DI application with transformations.
iam not getting any idea on this would you please help

1 ACCEPTED SOLUTION

Accepted Solutions
Patrick
Opal | Level 21

Below a fully working new cut of the %_sendmail() macro with all the parameters you've asked for. You know where it has to go.

When calling the macro only parameter TO is required.

/* send email
	 - attaches CSV if macro variable &attachFLG=1
*/
%macro _sendmail(
  to=,
  from=,
  sender=,
  cc=,
  subject=,
  importance=,
  attachCSV=&attachFLG
  );

  /* add single quotes around selected parameters */
  data _null_;
    length param_name $20 param_val $1000;
    do param_name='to','cc','importance';
      param_val=compbl(symget(param_name));
      if not missing(param_val) then
        do;
          param_val=cat("'",tranwrd(strip(param_val)," ","' '"),"'");
          call symputx(param_name,param_val);
          put param_name '=' param_val;
        end;
    end;
    stop;
  run;

  filename outbox email (&to)
    %if %nrbquote(&from) ne %nrbquote() %then from="&from";
    %if %nrbquote(&sender) ne %nrbquote() %then sender="%nrbquote(&sender)";
    %if %nrbquote(&cc) ne %nrbquote() %then cc=(&cc);
    %if %nrbquote(&subject) ne %nrbquote() %then subject="%unquote(%nrbquote(&subject))";
    %if %nrbquote(&importance) ne %nrbquote() %then 
      %do;
        %let importance=%upcase(&importance);
        %if   &importance='LOW'
          or  &importance='NORMAL'
          or  &importance='HIGH'
          %then 
            importance=&importance;
      %end;
    ;

    /* no attachements */
  %if &attachCSV=0 %then
    %do;
      data _null_;
        file outbox
/*          subject="Data from process - no attachments"*/
          ;
        put 'Folks,';
        put 'There is no data today';
        put 'This email sent without attachments';
      run;
    %end;

  /* with attachements */
  %else %if &attachCSV=1 %then
    %do;
      %local attachments;
      %let attachments=;
      %do i=1 %to &_INPUT_count;
        %let attachments=&attachments %unquote(%nrstr(%')&&_INPUT&i%nrstr(%'));
      %end;
      data _null_;
        file outbox
/*          subject="Data from process - with attachments"*/
          attach=(&attachments);
          ;
        put 'Folks,';
        put 'Attached the latest cut of data';
      run;
    %end;

  filename outbox clear;

%mend;
%_sendmail(
  to=dummy@dummy.com ,
  from=dummy@dummy.com,
  sender=dummy@dummy.com,
  cc=dummy@dummy.com,
  importance=HIGH,
  subject=your daily mail
  );

View solution in original post

33 REPLIES 33
LinusH
Tourmaline | Level 20

DI Studio have lot of good functionality.
Exporting external files migtht unfortenately not have that high qualoty transformations.

Given your description I'm not sure you an do it much better with other transformations.

What is the actual requirement?

If this is a feature that can repeat between jobs you may create a User Written Transformation to streamline your development/maintenance.

Data never sleeps
ErikLund_Jensen
Rhodochrosite | Level 12

Hi @JJP1 

 

DI Studio seems designed to work with tables, and file handling is implemented only through File writer and File reader transformations. It is not possible to connect a file to a standard transformation, not even a user written transformation, with one exception: The User Written Code transformation found under Transformations -> Data. 

 

The file must have a metadata registration as a prerequisite. The easiest way is to create an empty file in the output destination and register it as a delimited file without any columns - the warning about "no columns" can be ignored.

 

Then the file can be connected to the transformation as input, and if the defalut output work table is deleted, a file can be connected as output also, and the file names with full path are available in the macro variables &_INPUT and &_OUTPUT, which has two big anvantages when it comes to maintanance over the years:

  • No need for hard-coded file names in the code.
  • The files are included in impact analysis, so it is easy to check if a change in a table might have impact on the file. This works with variables too, if mappings are maintained in the user written code, even if the mappings are not used in code generation.

How can this help you:

 

You can use a User Written Code transformation, where you can make sure that the file doesn't exist physically if there is no data for it. Make two if you have more than one file. The transformation should also store the _&OUTPUT name in a global macro variable, so it is available later in the job. 

 

Then you can write another User Written Code transformation with the file as input and no output, which checks if the file exists and then send it as attachment. The disadvantage in your case is that the transformation can have only one file as input, but there is a way to overcome that too with the macro variables created in the "file writer" steps, because the mail transformation can use these variables in file checks and attachment strings instead of the local &_INPUT variable in the transformation.

 

So just connect one of the files as input to and include a yellow sticker in the job to tell that both files are used, so it is easy to tell what goes on when the job is opened, and check the Control Flow pane to ensure It looks that the mail transformation is executed after both files are written. 

 

It looks rather complicated, but it isn't in practice. I normally make it more complicated by always writing two files instead of one: The file with a fixed name that can be seen and opened from DI studio, and another file with a filestamp prefix that is actually sent, because then one can always go back in the physical files when there is any doubt about file content. It has saved me more than once that I could prove what was sent two months ago.

Patrick
Opal | Level 21

@JJP1 wrote:

Hi Team
There is a user written code written which is sending 2 csv files as an attachment and sending e-mails to few people.
if there is no data it will say message like "no data" in e-mail with out attachment.

The requiremnet here is i need to implement the above scenario by making use of SAS DI application with transformations.
iam not getting any idea on this would you please help


There isn't a ootb transformation for every specialty use case but there is a User Written transformation for once-off code and a Custom Transformation for implementation of re-usable code (used in multiple jobs). Use one of these two options and implement your own code.

If you select "user written body" then DIS will generate macros which contain info of Input and Output. Make use of these macros in your user written code so things become dynamic and you can change the input/output without having to change your user written code.

JJP1
Pyrite | Level 9

Thanks @ErikLund_Jensen & @Patrick ,

kindly find more details below :

currently 2 CSV files are being created using proc export code.

So i can keep this in user written code transformation.

currently we have email dataset where we are having column name as "To" and subject and content columns to send e-mail.

and we are making use of this dataset in SAS Di to send e-mails.

may i request to suggest how can i make use of this e-mail dataset and send the e-mail if it has no data and if it has no data just send no data with out attachment.sorry o trouble.is there any way like this pleas

 

Patrick
Opal | Level 21

@JJP1 wrote:

Thanks @ErikLund_Jensen & @Patrick ,

kindly find more details below :

currently 2 CSV files are being created using proc export code.

So i can keep this in user written code transformation.


From how it sounds this bit should be using the File Writer transformation and not user written code.

 


currently we have email dataset where we are having column name as "To" and subject and content columns to send e-mail.

and we are making use of this dataset in SAS Di to send e-mails.

may i request to suggest how can i make use of this e-mail dataset and send the e-mail if it has no data and if it has no data just send no data with out attachment.sorry o trouble.is there any way like this pleas

 


That sounds like something site specific so hard to tell how exactly you need to implement. But let's say you've used upstream the file writer transformation to create the two .csv and you've got these .csv registered as external file metadata. 

You could now have these two external file metadata objects as input to a user written code node. In doing so you would get something in the generated header code like %let _INPUT1 = <path and name of .csv>;

You know can write in the body section some custom code which checks if the file exists physically and then send an email accordingly. Many sites got already some autocall macro implemented (something like sendmail) so eventually you would just need a data step to populate parameters which you then use to call such a "sendmail" macro. If you don't have something like this then you would need to write it yourself. Here a starting point.

JJP1
Pyrite | Level 9

I was told that to make use of SAS DI functionality and create a SAS DI job for below code for e-mail part
note : Here .csv files that are created ,iam getting the data from work table that iam creating as part of this job.
would you please help

 

%macro sendmail;
data _null_;
call symputx('TEST',nobs);
set TEST_stats nobs=nobs;
run;
%if %eval(&TEST > 0) %then %do;
proc export
data=TEST_stats
outfile="%sysfunc(pathname(work))/TEST.csv" 
dbms=csv
label
replace
;
quit;
proc export
data=TEST1
outfile="%sysfunc(pathname(work))/TEST_TEST1.csv" 
dbms=csv
label
replace
;
quit;
filename email email
from="xxxx.com"
sender="xxxx.com"
to=("xxxx.com" "xxxx.com")
cc=("xxxx.com" "xxxx.com" "xxxx.com" "xxxx.com")
subject="Please refer attached TEST.csv and TEST_TEST1.csv for %sysfunc(putn(%eval(%sysfunc(today())-1),date9.))"
attach=("%sysfunc(pathname(work))/TEST.csv" "%sysfunc(pathname(work))/TEST_TEST1.csv")
importance='high'
;
data _null_;
file email;
put
"Hello" //
"Please refer attached TEST.csv and TEST_TEST1.csv for %sysfunc(putn(%eval(%sysfunc(today())-1),date9.))." //
;
put
"Test1" //
"Test2" //
;
run;
%end;
%else %do;
filename email1 email
from="xxxx.com"
sender="xxxx.com"
to=("xxxx.com" "xxxx.com")
cc=("xxxx.com" "xxxx.com" "xxxx.com" "xxxx.com")
subject="No data for TEST.csv and TEST_TEST1.csv"
importance='high'
;
Patrick
Opal | Level 21

Your %sendmail() macro shouldn't be doing anything else than sending emails. All the other stuff where you create attachments (or don't) and then decide what email text with/without attachments to send does NOT belong into the %sendmail() macro but into an upstream process (another node, eventually user written).

 

I would create a generic reusable %sendmail() macro implemented in DIS via a custom transformation with prompts for the parameters of the %sendmail() macro. 

Capture.JPG

@ScottBass published such %sendmail() macro code here which you could use. 

 

If using Scott's macro then your upstream process could simply populate as SAS table with the required parameter values and you then pass in the name of this table to macro parameter mail.

It's in this node where you populate this parameter table that you build the logic if you need to send an attachment or not - but that's just simple datastep logic very likely implemented as user written code.

JJP1
Pyrite | Level 9
Thanks Patrick ,
Now I have created two csv files separately using proc export user written
node.
Now I need to check in the same job for both the cab files whether it has
data in csv files or not.accordingly to send email
May I request to suggest how can I check this.can I do again 2 more
underwritten nodes to check for the data and one more node to send email?
Sorry to trouble you. Would you please help just higher level of nodes
require in sas Di job to achieve this please
Patrick
Opal | Level 21

With DIS you really should implement processes metadata driven and use ootb transformation whenever possible. 

With ETL processes you want stable data structures. Proc Export doesn't guarantee this. Is there any good reason why you can't use the File Writer transformation?

 

"check whether it has data in csv files or not"

Do you write a header line or is line 1 already data? 

The only way to figure out what you want is to read 1 line of data. Happy to give you pointer how that's done but please first explain why you can't use the File Writer and external file metadata for your file. Only once that sorted the next step can get designed.

 

And from an implementation point of view: It's easier to check if a table has zero rows than to check whether an external file has no data (given that there could be headers). So... is there any where clause on your export or would it suffices to check the rows in the source tables before the export?

 

JJP1
Pyrite | Level 9

Hi  @Patrick 
Sorry for my poor communication in previous message  that i was unable to convey you the help that i needed please.

please have a look on below  :

1.Currently i already created SAS DI half job where finally two datasets are getting created in work library. till this point I am getting output as expected and i am happy.

I need to proceed with below steps for above SAS DI job by making use of above 2 work datasets, which I am currently stuck on it.

 

Step 1 : I need to export the data from above 2 work datasets into 2 .csv files by making use of SAS Di transformations, may i request to suggest the best and efficient way to do this step in SAS DI please...

Step 2 : Send the e-mail to the users by attaching 2 .csv files only if it has data in both .csv files,if not send the e-mail with out attachment..to users.may i request to suggest the best and efficient way to do this step in SAS DI please...


Would you kindly suggest how can i achive step 1 and step 2 in SAS DI as efficient and best option and best approach please....

Sorry for miscommunication and interrupting and I am poor in understanding. would you please help..

 

Patrick
Opal | Level 21

OK - You'll get now in the following quite a bit more than what I normally would share in the forums here. But I acknowledge that Q&A with DIS jobs is a bit harder than talking SAS code only.

 

I've attached for you a fully executable self contained DIS job (version 4.903). If you select "My Folder" for import of the .spk then it will create the following sub-folder and objects.

Capture.JPG

 

You can use this job as template for what you need to implement. Below how the sample job looks like: 

Capture.JPG

 

For running/testing the sample job the only thing which requires change is the email address (hard coded in node 8).

Just replace in the code of node 8 "uw - send email" the dummy address with your own email address. Then you can execute the code and should receive an email with 2 attachments. 

Capture.JPG

 

To test the case where one of the source tables has no data: Go into node 1 and comment/uncomment code in order to execute the test case for "no data".

 

Once you're happy with what the template job does, replicate the relevant bits into your real job.

Important: For the current code to work the input tables to Node 3 "uw check" must be SAS tables (not views, not database). If that's not the case then you need to amend the code.

 

I'd recommend that you don't hard code email addresses into the code. Either use a macro variable instead which is defined somewhere in a config area or if hard coding use a group email address. Your goal must be that adding/removing recipients doesn't require any change to the DIS job.

 

SAS DIS best practice is to use out-of-the-box transformations before user written code. Even following this best practice I still had to use quite a bit of user written code. BUT: Even with user written code try to implement as metadata driven as you can. You'll see in the job posted that also the user written code uses metadata like DIS generated and metadata driven macro variable &_output for a target table instead of a hard coded table name.

 

On a side note:

I'd still prefer to have a reusable custom transformation for sending emails. You then would just have a step before this custom transformation which prepares the parameters (like sending an attachment or not) and then pass this to the reusable sendmail custom transformation (=replacing node 8 with a user written transformation preparing the parameters and a reusable custom transformation then sending the email).

Doing this here was just too much work so I went for a fully user written approach. 

 

I've had another look into the %sendmail macro code which I've proposed earlier for you to use as a starting point for implementation of a site specific %sendmail macro. I need to revoke this recommendation. This code is too involved and uses a whole framework of other utility macros which you also would need to implement. But: There is quite a bit of SAS code for sending emails published so I'm sure if implementing a generic %sendmail macro then a bit of Googling would allow you to not have to start from scratch.

 

I'm happy to answer more questions. My expectation though is that you now dive into the sample I've provided here and try to fully understand what it does and how it works before you ask follow-up questions.

 

And last but not least:

I'm writing the .csv to the SAS Work folder which only exists during a SAS session (path in external file metadata like: %sysfunc(pathname(work))/class.csv). I've done it this way to keep the whole DIS job self contained with a working path but not interfering with anything else you have in your environment. SAS will delete WORK as soon as you end your SAS session so there won't be any "left-overs". For your actual implementation: Consider writing the files to another disk location. Reason: Email delivery is not always guaranteed (i.e. email server down). So if this happens then you have still the .csv's available without re-running the job.

Also here: You need to implement in a way that no change to the DIS job is required when moving from one environment to the next. Make sure that the path you use will work for all environments (i.e. for different Lev's). I normally have a config where I define root paths in macro variables and then use in my code/DIS jobs paths like: &pathToProject/output

And about sending email attachments: If these attachments can get big then consider to zip them first and attach the zip archive (filename zip or ods package lets you create such zip archives using SAS).

JJP1
Pyrite | Level 9

Thanks @Patrick for your kind response and helping me to proceed further on this,

Yes i follow the instructions provided by you and i am very much thankful to you for your patience and kindness.

Patrick
Opal | Level 21

Let me know how you go. Now that I've got invested that much I'm also curious if that helped and allowed you to succeed.

JJP1
Pyrite | Level 9

Yes thanks @Patrick .

Iam trying to import package ,but iam getting attached error.iam using SAS DI 4.4Error_SASDI.PNG

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

How to connect to databases in SAS Viya

Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 33 replies
  • 7290 views
  • 3 likes
  • 4 in conversation