BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
KOUPGANG
Calcite | Level 5

Hi all,

Please i think i need help here.

I Have a proc sql code running for more than 4 hours to give me an output.

the dataset on the left (Base0) has only 1 million rows, but Base2 is a remote dataset in a global repository.

PLEASE why does it takes so long to execute (4 hours) ?

How can it be optimized ?

NB Note than the key variable doesnt have the same structure in both dataset.

Thanks

 

proc sql;

create table lib1.Base1 as

select a.*, b.DW_DATE_KEY, b.SERVICE_CLASS,

b.TOTAL_VOICE_AMT,b.TOTAL_DURATION,

b.NB_CALLS, b.NB_CALLS_ONNET, b.NB_CALLS_OFFNET,b.NB_CALLS_INTERNATIONAL,

b.VOICE_AMT_ONNET,b.VOICE_AMT_OFFNET,b.VOICE_AMT_INTERNATIONAL,

b.VOICE_DUREE_ONNET,b.VOICE_DUREE_OFFNET , b.VOICE_DUREE_INTERNATIONAL,

b.MA_VOICE_USED, b.BONUS_VOICE_USED

from lib1.Base0 as a

left join lib2.Base1(where =(DW_DATE_KEY >= 20160516 and DW_DATE_KEY <=20160630)) as b

on a.msisdn =(input(b.msisdn, best12.)-237000000000);

quit;

1 ACCEPTED SOLUTION

Accepted Solutions
Doc_Duke
Rhodochrosite | Level 12

A guess that should help.

 

Copy the subset of lib2 to WORK and do the ID transform during the copy.  The left join may be making multiple passes through lib2 and input is not the most efficient function in SAS.

View solution in original post

4 REPLIES 4
Doc_Duke
Rhodochrosite | Level 12

A guess that should help.

 

Copy the subset of lib2 to WORK and do the ID transform during the copy.  The left join may be making multiple passes through lib2 and input is not the most efficient function in SAS.

Kurt_Bremser
Super User

I'd fetch the base2 table to a local SAS dataset in a separate step. In that step, do the calculation for your key. Then sort per the key, which should make things much easier for the SQL step.

You might still find that proc sort and a data step merge are faster. If you have a many-to many relationship requiring a cartesian join, you must use SQL, of course.

KOUPGANG
Calcite | Level 5
Thanks a lot. Its faster whith Base2 dataset in Work library.
LinusH
Tourmaline | Level 20
Options msglevel = i sastrace = ',,,d' sastraceloc = saslog;
Helps you understand what gets passed to the underlying dbms.
Try to avoid function calls in filter/join criterias.
How many records does base2 have?
How many records typically meet the date fire criteria?
Any indexes on dw_date_key or msisdn?
Data never sleeps

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 4 replies
  • 1648 views
  • 0 likes
  • 4 in conversation