BookmarkSubscribeRSS Feed
qwererty
Calcite | Level 5

Hi,

I am trying to utilise the type 1 table loader so I can update a table with only changed dimensions.  The table has 70k rows and 5 variables.  I require 3 varibale s as my lookup to check for any change and then I have one target column that is changed.  This is the first time I have  used the a SCD and I am finding it can take up to 15 miutes to run with potentially only 3k records changing.  Firstly is this normal? Secondly how can I optimise this time?  I have experimented with different hash lookup sizes and no noticeable improvement is made.

Any ideas?

Thanks

1 REPLY 1
LinusH
Tourmaline | Level 20

Sounds like a heck of time for so little records, independent of update technique.

And yes, there are stuff you can do to optimize the code.

But first things first. Locate in the log the step(s) that consumes the most resources, and try to start from there.

What aboit other processing - is this the only job that performs poor? Or coudl it be a system constraint?

Why are you using the Type 1 Loader? I seldom use it. Depending on your requirement you could use a standard Table Loader, or the Type 2 Loader.

Data never sleeps

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 1 reply
  • 802 views
  • 0 likes
  • 2 in conversation