topic Re: SAS Dataflux Batch Job Transformation - Lineage in SAS Data Management

SAS Dataflux Batch Job Transformation - Lineage

sanalitics — Wed, 04 Apr 2018 13:40:16 GMT

Hi All, In SAS DI, do we have a way, where we could have an input table and output tables connected to the Dataflux Batch Job Transformation? Since we don't have nodes option in Dataflux Batch Job Transformation available by default do you see any work around to maintain lineage? Thanks, Sandeep

Re: SAS Dataflux Batch Job Transformation - Lineage

RonAgresta — Thu, 12 Apr 2018 16:23:31 GMT

Unfortunately this is not possible to do automatically. That transformation calls out to a different server. The jobs in the DataFlux environment can contribute to the Lineage repository though. You would need to use DI Studio and and some API calls and SQL code to connect the DataFlux metadata objects to SAS object metadata in the relationship repository that SAS Lineage uses.

Ron

Re: SAS Dataflux Batch Job Transformation - Lineage

sanalitics — Thu, 12 Apr 2018 19:32:09 GMT

Hi Ron,

I was also thinking of a user written transformation that could replace the batch job transformation in DI...

But the challenge is to get the connection to the DataMangagement server and its jobs in the options in order to select these from the drop down... any thoughts on this?

Best Regards,

Sandeep

Re: SAS Dataflux Batch Job Transformation - Lineage

RonAgresta — Thu, 12 Apr 2018 19:56:37 GMT

It is possible to make web service calls to DM Server for things like lists of batch jobs. The DM Server documentation covers some of this. So maybe a custom transform using web service calls could help. I don't think you can get the connection information though in the same way.

Re: SAS Dataflux Batch Job Transformation - Lineage

sanalitics — Fri, 20 Apr 2018 13:14:58 GMT

Do you think we should ask SAS to provide a solution in such cases; where we could maintain the lineage?? If you know a way where we could approach SAS to look into this... could you forward to its R&D ?

Re: SAS Dataflux Batch Job Transformation - Lineage

RonAgresta — Mon, 23 Apr 2018 10:46:18 GMT

I can take this as a requirement for the R&D team. This is part of my job here at SAS.

Ron