how to remove duplicates in SAS Enterprise Miner

MoeYousefi — Sun, 01 Apr 2018 04:40:49 GMT

Hi All,

I'm fairly new to SAS E-Miner and was just wondering if you could help me out with my query of " how to eliminate duplicated records in SAS E-Miner?"

Many thanks,

Moe

Re: how to remove duplicates in SAS Enterprise Miner

MikeStockstill — Mon, 02 Apr 2018 17:05:56 GMT

Hello MoeYousefi-

Enterprise Miner does not have specific functionality for removing duplicate observations. However, you can run a SAS Code node and invoke PROC SORT with the NODUPKEY option.

Example:

- Add a SAS Code node to your flow.

- Select the Code Editor property. Enter code like this:

proc sort nodupkey data=&EM_IMPORT_DATA out=&EM_EXPORT_TRAIN;
var < list of variables that define unique vs duplicate >;
run;

- Close the node. Run the node. Continue your flow.

The NODUPKEY option tells PROC SORT to keep only unique rows as defined by the variables on the VAR statement.

&EM_IMPORT_DATA is a SAS Code node macro variable that resolves to the data source that is coming in to the SAS Code node.

&EM_EXPORT_TRAIN is a SAS Code node macro variable that resolves to the data source that is created by the SAS Code node.

There is no real advantage to running PROC SORT in a SAS Code node in this specific scenario. In fact, you might be better served by running PROC SORT in the coding job that prepares the data set for use in Enterprise Miner.

Have a great week!

Re: how to remove duplicates in SAS Enterprise Miner

MoeYousefi — Tue, 03 Apr 2018 12:09:11 GMT

Thank you so much Mike,

Much appreciated.

topic Re: how to remove duplicates in SAS Enterprise Miner in SAS Data Science

how to remove duplicates in SAS Enterprise Miner

Re: how to remove duplicates in SAS Enterprise Miner

Re: how to remove duplicates in SAS Enterprise Miner