BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
twildone
Pyrite | Level 9

Hi, I have created a dataset from two different datasets and I want to remove any duplicates in the new datasets. I have tried using the the proc sql distinct approach as well as a Data step with the first. approach and I keep getting a dataset that has duplicates but some of the duplicates are removed.. The dataset has only 3 variable, ID as a character variable and Unit Price and Effective Date both as numeric variables. I double checked and made sure are and were formatted the same before creating the new dataset. Any suggestion on what I may be missing and should check....Thanks in Avance.

1 ACCEPTED SOLUTION

Accepted Solutions
ballardw
Super User

I would be interested is seeing the SQL code that did not work. Order of when to appear DISTINCT may have been the issue.

Also you may need to look at the ID in case some of them have one or more leading blank characters as "string" is not equal to " string" and could cause apparent duplicates.

View solution in original post

3 REPLIES 3
Reeza
Super User

Your join may have issues. 

Otherwise you can check unique/duplicate records in SAS with NOUNIQUEKEY in proc sort. Available in SAS 9.3+

proc sort data=have out=duprec nouniquekey uniqueout=want;
by ID;
run;

ballardw
Super User

I would be interested is seeing the SQL code that did not work. Order of when to appear DISTINCT may have been the issue.

Also you may need to look at the ID in case some of them have one or more leading blank characters as "string" is not equal to " string" and could cause apparent duplicates.

twildone
Pyrite | Level 9


Hi Reeza & Ballardw,

Thanks for your help and suggestions. With your suggestions of the possibility of blank characters for the ID variable, it turns out that there must have been a blank or blanks in the Unit Price as I used the Compress Function on both the ID and Unit Price variables having to convert Unit Price to a character variable first. It did the trick. Thanks once againn for your help.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 3 replies
  • 744 views
  • 3 likes
  • 3 in conversation