BookmarkSubscribeRSS Feed
qwererty
Calcite | Level 5

Hi,

I am trying to utilise the type 1 table loader so I can update a table with only changed dimensions.  The table has 70k rows and 5 variables.  I require 3 varibale s as my lookup to check for any change and then I have one target column that is changed.  This is the first time I have  used the a SCD and I am finding it can take up to 15 miutes to run with potentially only 3k records changing.  Firstly is this normal? Secondly how can I optimise this time?  I have experimented with different hash lookup sizes and no noticeable improvement is made.

Any ideas?

Thanks

1 REPLY 1
LinusH
Tourmaline | Level 20

Sounds like a heck of time for so little records, independent of update technique.

And yes, there are stuff you can do to optimize the code.

But first things first. Locate in the log the step(s) that consumes the most resources, and try to start from there.

What aboit other processing - is this the only job that performs poor? Or coudl it be a system constraint?

Why are you using the Type 1 Loader? I seldom use it. Depending on your requirement you could use a standard Table Loader, or the Type 2 Loader.

Data never sleeps

hackathon24-white-horiz.png

The 2025 SAS Hackathon has begun!

It's finally time to hack! Remember to visit the SAS Hacker's Hub regularly for news and updates.

Latest Updates

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

SAS Training: Just a Click Away

 Ready to level-up your skills? Choose your own adventure.

Browse our catalog!

Discussion stats
  • 1 reply
  • 1208 views
  • 0 likes
  • 2 in conversation