SAS Data Integration Studio, DataFlux Data Management Studio, SAS/ACCESS, SAS Data Loader for Hadoop and others

Regarding constraint conditions for the DI Studio table loader transformation

Accepted Solution Solved
Reply
Frequent Contributor
Posts: 90
Accepted Solution

Regarding constraint conditions for the DI Studio table loader transformation

Regarding the Table Loader transformation in DI Studio. I have some difficulty understanding when the constraint conditions should be set to Before Load, and when they should be set to After Load. Are any of you able to explain this to me, or give me a good rule-of-thumb about it? I know the job runs much quicker when the constraint conditions are set After Load, but I don't want to risk anything by always doing it that way. So basically, when should I set constraint conditions Before Load, and why?

Thanks for your time. Smiley Happy


Accepted Solutions
Solution
‎01-17-2013 06:25 AM
Super User
Posts: 5,431

Re: Regarding constraint conditions for the DI Studio table loader transformation

Posted in reply to EinarRoed

Hard to give a rule of thumb, it all depends on your data, data cleansing, data model etc.

How do you intend to use constraints?

If you use the ETL process to cleanse your data in most of the aspects that you can think, any constraint will be used as extra security, and not often triggered during load. Then it could be wise to apply constraints after load.

What you risk is by applying after load, is that non consistent data may be loaded/updated in the target table, and you need to figure out how to roll those inserts/updates out.

Another aspect is data volume vs the load window, is performance an issue?

If not, you could have constraints applied permanently.

Data never sleeps

View solution in original post


All Replies
Solution
‎01-17-2013 06:25 AM
Super User
Posts: 5,431

Re: Regarding constraint conditions for the DI Studio table loader transformation

Posted in reply to EinarRoed

Hard to give a rule of thumb, it all depends on your data, data cleansing, data model etc.

How do you intend to use constraints?

If you use the ETL process to cleanse your data in most of the aspects that you can think, any constraint will be used as extra security, and not often triggered during load. Then it could be wise to apply constraints after load.

What you risk is by applying after load, is that non consistent data may be loaded/updated in the target table, and you need to figure out how to roll those inserts/updates out.

Another aspect is data volume vs the load window, is performance an issue?

If not, you could have constraints applied permanently.

Data never sleeps
🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 1 reply
  • 396 views
  • 0 likes
  • 2 in conversation