DATA Step, Macro, Functions and more

Most efficient way to merge multiple datasets

Reply
Regular Contributor
Posts: 150

Most efficient way to merge multiple datasets

I have 20 datasets each with 200,000 rows and anywhere from 30-70 columns. I can use a data step to merge the files by ID but is there a more efficient way to do this? I tried to merge using a left outer join in Proc SQL but that keeps giving me errors. Any suggestion sare appreciated.

 

 

Super User
Posts: 17,819

Re: Most efficient way to merge multiple datasets

Are you looking for efficiency in terms of computer processing or in terms of minimum programming?

Regular Contributor
Posts: 150

Re: Most efficient way to merge multiple datasets

A combo of both but more important I think is programming since we don't have to complete the merge many times. Thank you!

Super User
Posts: 10,497

Re: Most efficient way to merge multiple datasets

You should show code and the errors from the log. Best is to post the copied log/ error message into a code box opened with the forum {i} menu icon to preserve the formatting of the error messages. Many error messages have indicators as to where on a line the error occurs and the main message windows reformat log results.

 

IF any of the error relate to mismatched variable types, or different lengths of character, then you regardless of approach you will need to address that before attempting to merge the data.

 

 

Ask a Question
Discussion stats
  • 3 replies
  • 107 views
  • 0 likes
  • 3 in conversation