topic Efficient Proc Sql Joining in SAS Programming

Efficient Proc Sql Joining

AZIQ1 — Mon, 14 Aug 2017 15:39:22 GMT

Hi,

Trying to create a left outer join with a table on the server. TableB is on the server and is a huge table, and TableA is the local table:

PROC SQL;

CREATE TABLE NEW AS

SELECT a.*,

b.VAR1, b.VAR2, b.VAR3, b.VAR4, b.VAR5

FROM TableA AS a

LEFT OUTER JOIN TABLEB AS b

ON a.ID = b.ID;

QUIT;

This code is taking very long actually more than 2 hours. Is there an efficient way I should run this code? like creating a temp table from the server and using that to join TableA.

Thank you

Re: Efficient Proc Sql Joining

Reeza — Mon, 14 Aug 2017 15:44:24 GMT

How big is Table A?

If it's smaller, you can consider creating macro variable that has the values you want and then you can filter it first and then join once it's been extracted.

proc sql;

create tableB_Sub as

select b.VAR1, b.VAR2, b.VAR3, b.VAR4, b.VAR5

from tableB as b

where ID in ( LIST OF IDS HERE);

quit;

Then join afterwards.

Re: Efficient Proc Sql Joining

AZIQ1 — Mon, 14 Aug 2017 16:24:34 GMT

Thank you this worked.

Re: Efficient Proc Sql Joining

utrocketeng — Mon, 14 Aug 2017 16:26:52 GMT

Have you validated that there are indexes on the columns you are joining on (a.ID and b.ID)? In my experience, indexes are critical for efficient queries.

Re: Efficient Proc Sql Joining

Reeza — Mon, 14 Aug 2017 17:05:08 GMT

When joining a table from the server with a local table SAS firsts brings down the entire table from the server and does the work locally. This is inefficient, so first subsetting the table and bringing it down helps to speed it up.

Re: Efficient Proc Sql Joining

utrocketeng — Wed, 16 Aug 2017 11:54:07 GMT

Reeza,

For the reason you described, i have become a proponet within my company to leverage pass through PROC SQL queries. if the table hosted within SAS is reasonably small, i will write it up to the remote server as a temp table then do the join with that db.