topic Re: Duplicates records into separate dataset. in SAS Enterprise Guide

Duplicates records into separate dataset.

brm — Mon, 06 Jun 2011 20:29:52 GMT

Hi,

I trying to keep duplicate records in sepate dataset.I' doing it like this,but having problem with large data.Can anybody suggest how can i do this in other
way.

proc freq data = b noprint ;
/*by DRC_CASE_ID Milestone_Date Product MILESTONE_TYPE ;*/
table DRC_CASE_ID*Milestone_Date*Product*MILESTONE_TYPE*DAYS_OF_THERAPY / out =
test18(drop=percent where = (Count >1));
run;

ERROR: The requested table is too large to process.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: There were 219548 observations read from the data set WORK.B.

Thanks,
brm

Re: Duplicates records into separate dataset.

Patrick — Mon, 06 Jun 2011 21:25:29 GMT

data have;
a=1;b=1;c=1; output;
a=1;b=2;c=1; output;
a=2;b=2;c=1; output;
a=1;b=2;c=1; output;
a=1;b=2;c=1; output;
run;

/* for duplicates: Keeps the first rec in want1, moves all other recs to DupsWant1 */
proc sort data=have out=want1 dupout=DupsWant1 noduprec;
by a b c;
run;

/* Creates a table with duplicates, a table with all obs having no duplicates, and a table with all de-duped obs */
proc sql;
create table Duplicates as
select *, count(*) as N_Dups
from have
group by a,b,c
having count(*)>1
;

create table ObsWithNoDuplicates as
select *
from have
group by a,b,c
having count(*)=1
;

create table AllObsDeDuped as
select DISTINCT *
from have
;

quit;

HTH
Patrick

Re: Duplicates records into separate dataset.

brm — Tue, 07 Jun 2011 22:42:14 GMT

Thanks a lot....those worked fine for me.