topic Re: How SAS Custom Macro make feature engineering easier ( Outlier Detection, One-Hot Encoding) in SAS Programming

How SAS Custom Macro make feature engineering easier ( Outlier Detection, One-Hot Encoding)

SurajSaini — Wed, 20 Jan 2021 12:33:55 GMT

https://seleritysas.com/blog/2020/12/10/sas-custom-macros-that-make-feature-engineering-easy-for-data-scientists-data-engineers-and-machine-learning-specialists/

One Hot Encoding in Base SAS
Outlier Detection in Base SAS
Define Dataset in Base SAS
Lag Features in Base SAS

Re: How SAS Custom Macro make feature engineering easier ( Outlier Detection, One-Hot Encoding)

PaigeMiller — Wed, 20 Jan 2021 12:39:29 GMT

While I would love to have some automated outlier detection code that I didn't have to write, you present code that can't be copied for my use ... so my question: is there a place where the actual text of these SAS macros can be downloaded from?

Re: How SAS Custom Macro make feature engineering easier ( Outlier Detection, One-Hot Encoding)

SurajSaini — Wed, 20 Jan 2021 12:46:17 GMT

/* Macro def */
%macro outliers(dat,var);
options nonotes;
proc univariate data=&dat normal noprint;
var &var;
output out=ttest normaltest=Test probn=P_Value;
run;
Data _Null_;
set ttest;
%if P_value > 0.05 %Then %do;
option notes;
%put NOTE: &var is normally distributed hence it select STD method to find Outliers.;
%put NOTE: You can check statistics and pvalue in work.ttest table;
options nonotes;
Proc SQL noprint;
Select Mean(&var)
into: me
from &dat;
select std(&var)
into:sd
from &dat;
quit;
run;
Data outlier;
set &dat;
%Let Min_cutoff= %sysevalf(&me - (3* &sd));
%Let Max_cutoff= %sysevalf(&me + (3* &sd));
where &var < &Min_cutoff or &var > &Max_cutoff;
run;
%end;
%else %do;
options notes;
%put NOTE: &var is not normally distributed hence it select percentile method to find Outliers.;
%put NOTE: You can check statistics and pvalue in work.ttest table & percentile values in work.ranges table;
options nonotes;
proc means data=&dat stackods n qrange p1 p99 ;
var &var;
ods output summary=ranges;
run;
proc sql noprint;
select P1 into:Min
from ranges;
select P99 into : Max
from Ranges;
quit;
run;
Data outliers;
set &dat;
Where &var < &Min or &var > &Max;
run;
%end;
options notes;
%mend;

/* Check Macro */
*options nomprint nomlogic nosymbolgen;
%outliers(Lib.sas dataset, Variable-Name)

Re: How SAS Custom Macro make feature engineering easier ( Outlier Detection, One-Hot Encoding)

SurajSaini — Wed, 20 Jan 2021 12:50:51 GMT

I will share a GitHub link as soon as possible

Re: How SAS Custom Macro make feature engineering easier ( Outlier Detection, One-Hot Encoding)

PaigeMiller — Wed, 20 Jan 2021 12:51:36 GMT

Thanks!

Re: How SAS Custom Macro make feature engineering easier ( Outlier Detection, One-Hot Encoding)

Ksharp — Wed, 20 Jan 2021 13:19:49 GMT

calling @Rick_SAS

Re: How SAS Custom Macro make feature engineering easier ( Outlier Detection, One-Hot Encoding)

SurajSaini — Sun, 24 Jan 2021 20:01:56 GMT

You can download code from here

https://github.com/Suraj-617/Blogs/blob/master/How%20SAS%20Macro%20Makes%20Features%20Engineering%20Easier