BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
SASNE
Obsidian | Level 7

Hi Gurus,

 

I have a situration where the SQL queue has some joining , do you think if this is a solution if i retreive data from each table and merge it in SAS locally instead of sending join queue to Oracle?

 

in regards to performance issue if the dataset is around 50GB.

 

Thanks

 

 

1 ACCEPTED SOLUTION

Accepted Solutions
Tom
Super User Tom
Super User

In general it will be faster to join in the source database rather than extracting.  Moving 50Gb of data would take a long time.

 

The only exception would be is if your join is essentially replicating data such that the result of the join is much larger than just pulling the source tables individually.  For example if the join was to convert a small code variable in your large fact table into a long descriptive text by joining with a dimension (lookup, format) table. In that case it might be faster post pone the lookup. Instead just keep the coded value in the large table so that less data needs to move from Oracle to SAS.  Then you could convert the code to the description in SAS.  For example you could convert the dimension table to a FORMAT and attach it to the variable with coded value.

View solution in original post

2 REPLIES 2
Tom
Super User Tom
Super User

In general it will be faster to join in the source database rather than extracting.  Moving 50Gb of data would take a long time.

 

The only exception would be is if your join is essentially replicating data such that the result of the join is much larger than just pulling the source tables individually.  For example if the join was to convert a small code variable in your large fact table into a long descriptive text by joining with a dimension (lookup, format) table. In that case it might be faster post pone the lookup. Instead just keep the coded value in the large table so that less data needs to move from Oracle to SAS.  Then you could convert the code to the description in SAS.  For example you could convert the dimension table to a FORMAT and attach it to the variable with coded value.

SASNE
Obsidian | Level 7

thx Tom

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 2 replies
  • 749 views
  • 2 likes
  • 2 in conversation