Decomposition and Orchestration in DI Studio and LSF.

[AngusLooney]

As part of the UK and Ireland SAS User Group - https://www.meetup.com/SUGUKI/ - I volunteered to give talk on " Decomposition and Orchestration in DI Studio and LSF".

Posting the slides here as well in case anyone finds them useful, thought or comment provoking - or perhaps to kick off an discussion.

The main point I was hoping to make was the features provided by LSF or equivalent offerings should be viewed as an intrinsic part of the toolset, and that they promote, support and enable the adoption of fundemental approaches to how ETL processes are designed, built and operated. "Zen and the Art of DI Studio" - almost.

It's an independant user group, so the slides are my own thoughts/ideas/opinions, in no way "SAS official" or anything - hence the non-corporate typography.

Enjoy.

[LaurieF]

That's bloody excellent. Thank you. I've come to the same, or similar, conclusion from a slightly different direction: a DI job should do one thing (have one target), and that makes it easy to put into an LSF stream. LSF can then do its magic by parallelising as much as possible, and we can all go off to the pub.

[AngusLooney]

Absolutely. If this was useful, there are a couple of other decks (and a video) from me hereabouts on related topics, not to blow my own trumpet...

[KumarT_SAS]

Interesting and a great process to combine the mix of data management tools. If we look at SAS Grid Manager with LSF, we are looking at a more advanced version of the scheduling process, parallelism and most importantly load balancing. Just LSF with SAS DI Studio, definitely looks lean and less complex for most of the repeatable data management jobs. 

[AngusLooney]

Single server LSF is just the same "LSF Grid" stuff as a full grid, just on a single server.