DATA Step, Macro, Functions and more

Efficient Proc Sql Joining

Accepted Solution Solved
Reply
Contributor
Posts: 71
Accepted Solution

Efficient Proc Sql Joining

Hi,

Trying to create a left outer join with a table on the server. TableB is on the server and is a huge table, and TableA is the local table:

 

 

PROC SQL;

CREATE TABLE NEW AS

SELECT a.*,

b.VAR1, b.VAR2, b.VAR3, b.VAR4, b.VAR5

FROM TableA AS a

LEFT OUTER JOIN TABLEB AS b

ON a.ID = b.ID;

QUIT;

 

 

This code is taking very long actually more than 2 hours. Is there an efficient way I should run this code? like creating a temp table from the server and using that to join TableA.

 

Thank you

 


Accepted Solutions
Solution
‎08-14-2017 12:24 PM
Super User
Posts: 19,815

Re: Efficient Proc Sql Joining

How big is Table A?

If it's smaller, you can consider creating macro variable that has the values you want and then you can filter it first and then join once it's been extracted.

 

proc sql;

create tableB_Sub as

select b.VAR1, b.VAR2, b.VAR3, b.VAR4, b.VAR5

from tableB as b

where ID in ( LIST OF IDS HERE);

quit;

 

Then join afterwards.

View solution in original post


All Replies
Solution
‎08-14-2017 12:24 PM
Super User
Posts: 19,815

Re: Efficient Proc Sql Joining

How big is Table A?

If it's smaller, you can consider creating macro variable that has the values you want and then you can filter it first and then join once it's been extracted.

 

proc sql;

create tableB_Sub as

select b.VAR1, b.VAR2, b.VAR3, b.VAR4, b.VAR5

from tableB as b

where ID in ( LIST OF IDS HERE);

quit;

 

Then join afterwards.

Contributor
Posts: 71

Re: Efficient Proc Sql Joining

Thank you this worked.
Occasional Contributor
Posts: 6

Re: Efficient Proc Sql Joining

Have you validated that there are indexes on the columns you are joining on (a.ID and b.ID)?  In my experience, indexes are critical for efficient queries.

Super User
Posts: 19,815

Re: Efficient Proc Sql Joining

Posted in reply to utrocketengineer

When joining a table from the server with a local table SAS firsts brings down the entire table from the server and does the work locally. This is inefficient, so first subsetting the table and bringing it down helps to speed it up. 

 

Occasional Contributor
Posts: 6

Re: Efficient Proc Sql Joining

Reeza,

 

For the reason you described, i have become a proponet within my company to leverage pass through PROC SQL queries.  if the table hosted within SAS is reasonably small, i will write it up to the remote server as a temp table then do the join with that db.

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 5 replies
  • 90 views
  • 3 likes
  • 3 in conversation