Help using Base SAS procedures

Proc SQL with left join clause so long in execution.

Accepted Solution Solved
Reply
New Contributor
Posts: 2
Accepted Solution

Proc SQL with left join clause so long in execution.

Hi all,

Please i think i need help here.

I Have a proc sql code running for more than 4 hours to give me an output.

the dataset on the left (Base0) has only 1 million rows, but Base2 is a remote dataset in a global repository.

PLEASE why does it takes so long to execute (4 hours) ?

How can it be optimized ?

NB Note than the key variable doesnt have the same structure in both dataset.

Thanks

 

proc sql;

create table lib1.Base1 as

select a.*, b.DW_DATE_KEY, b.SERVICE_CLASS,

b.TOTAL_VOICE_AMT,b.TOTAL_DURATION,

b.NB_CALLS, b.NB_CALLS_ONNET, b.NB_CALLS_OFFNET,b.NB_CALLS_INTERNATIONAL,

b.VOICE_AMT_ONNET,b.VOICE_AMT_OFFNET,b.VOICE_AMT_INTERNATIONAL,

b.VOICE_DUREE_ONNET,b.VOICE_DUREE_OFFNET , b.VOICE_DUREE_INTERNATIONAL,

b.MA_VOICE_USED, b.BONUS_VOICE_USED

from lib1.Base0 as a

left join lib2.Base1(where =(DW_DATE_KEY >= 20160516 and DW_DATE_KEY <=20160630)) as b

on a.msisdn =(input(b.msisdn, best12.)-237000000000);

quit;


Accepted Solutions
Solution
‎07-07-2016 08:04 AM
Trusted Advisor
Posts: 2,116

Re: Proc SQL with left join clause so long in execution.

A guess that should help.

 

Copy the subset of lib2 to WORK and do the ID transform during the copy.  The left join may be making multiple passes through lib2 and input is not the most efficient function in SAS.

View solution in original post


All Replies
Solution
‎07-07-2016 08:04 AM
Trusted Advisor
Posts: 2,116

Re: Proc SQL with left join clause so long in execution.

A guess that should help.

 

Copy the subset of lib2 to WORK and do the ID transform during the copy.  The left join may be making multiple passes through lib2 and input is not the most efficient function in SAS.

Super User
Posts: 7,843

Re: Proc SQL with left join clause so long in execution.

I'd fetch the base2 table to a local SAS dataset in a separate step. In that step, do the calculation for your key. Then sort per the key, which should make things much easier for the SQL step.

You might still find that proc sort and a data step merge are faster. If you have a many-to many relationship requiring a cartesian join, you must use SQL, of course.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
New Contributor
Posts: 2

Re: Proc SQL with left join clause so long in execution.

Posted in reply to KurtBremser
Thanks a lot. Its faster whith Base2 dataset in Work library.
Super User
Posts: 5,435

Re: Proc SQL with left join clause so long in execution.

Options msglevel = i sastrace = ',,,d' sastraceloc = saslog;
Helps you understand what gets passed to the underlying dbms.
Try to avoid function calls in filter/join criterias.
How many records does base2 have?
How many records typically meet the date fire criteria?
Any indexes on dw_date_key or msisdn?
Data never sleeps
☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 4 replies
  • 258 views
  • 0 likes
  • 4 in conversation