topic Re: proc sort data using noduprecs in New SAS User

proc sort data using noduprecs

veda8 — Mon, 20 Jul 2020 14:08:49 GMT

do we have to always sort by _ALL_ to use noduprecs?

Re: proc sort data using noduprecs

PeterClemmensen — Mon, 20 Jul 2020 14:16:33 GMT

No:

data have;
input ID var;
datalines;
1 10
1 20
1 10
3 50
3 50
3 50
2 30
2 30
2 40
;

proc sort data=have noduprec;
   by ID;
run;

Re: proc sort data using noduprecs and no dupkey

veda8 — Mon, 20 Jul 2020 14:31:57 GMT

when use nodupkey and give two variables in dupkey

eg :

by id var;

which var(s) is considered as dupkey?

Re: proc sort data using noduprecs and no dupkey

ballardw — Mon, 20 Jul 2020 14:48:28 GMT

@veda8 wrote:

when use nodupkey and give two variables in dupkey

eg :

by id var;

which var(s) is considered as dupkey?

Both.

The "key" is whatever is on the By statement.

NODUPKEY

checks for and eliminates observations with duplicate BY values. If you specify this option, then PROC SORT compares all BY values for each observation to the ones for the previous observation that is written to the output data set. If an exact match is found, then the observation is not written to the output data set.

Re: proc sort data using noduprecs

Reeza — Mon, 20 Jul 2020 16:37:40 GMT

You have to use a double sort with NODUPRECS as well. It isn't supported any longer and you shouldn't use it in production code going forward. Use NODUPKEY instead. SAS takes a while to deprecate features but it has been removed from the documentation and is currently maintained for backwards compatibility.

https://documentation.sas.com/?docsetId=proc&docsetVersion=9.4&docsetTarget=p02bhn81rn4u64n1b6l00ftdnxge.htm&locale=en

Re: proc sort data using noduprecs

Ksharp — Tue, 21 Jul 2020 13:12:34 GMT

I think it is Yes.
And better NOT use noduprecs , try nodupkey + _all_ instead.

Re: proc sort data using noduprecs

Tom — Tue, 21 Jul 2020 13:37:11 GMT

@veda8 wrote:

do we have to always sort by _ALL_ to use noduprecs?

You don't "have" to, SAS will happily let you use any subset of the variables in the BY statement.

But if you want the result to eliminate all duplicate records you do.

The reason is that the DUP check (or as it has been renamed the DUPRECS check) only compares adjacent records. So if you only sort by a subset of the variables then it is possible for two records that are exactly the same to be output. They just need at least one observation that is different on some non-key (by) variable in between them.