topic Re: Process Video & Picture file in SAS DI (ETL job) in Microsoft Integration with SAS

Process Video & Picture file in SAS DI (ETL job)

anandsasbi — Thu, 10 Oct 2019 08:49:23 GMT

can we import export Video & Picture unstructured file in SAS DI Studio . I am working on RFP and client having Video & Image file as source data which need to load on Azure data lake (HDFS file system ) & Azure DWH (SQL Server ) using SAS ETL job . I want to know that can we process Video & Picture unstructured file in SAS DI Studio if use than which transformation and format need to use . can we store Video & Picture unstructured data in SAS work table.

Re: Process Video & Picture file in SAS DI (ETL job)

Kurt_Bremser — Thu, 10 Oct 2019 08:52:05 GMT

Short answer: no.

The maximum length of unstructured data you can store in SAS is 32767 bytes (maximum length of character variables).

You need to set up a repository for such files, and only store the path and filenames in the SAS dataset.

Re: Process Video & Picture file in SAS DI (ETL job)

Patrick — Thu, 10 Oct 2019 13:50:16 GMT

@anandsasbi

So if I get this right then you have somewhere Video & Picture files (BLOB) which you need to move to Azure Blob storage surfaced via SQL Server.

SAS 9.4 can't store Blob in a SAS table (SAS Viya/CAS could potentially). But.... I don't think it makes a lot of sense in first place to load data from some source to Azure via some intermediary SAS storage.

You still can use SAS as "remote control" to orchestrate the process. You can implement such user written SAS code in DIS via a user written transformations or re-usable via custom transformations. The only ootb transformation which is eventually of use is Transfer to Hadoop.

Re: Process Video & Picture file in SAS DI (ETL job)

SASKiwi — Thu, 10 Oct 2019 23:01:50 GMT

This is a bit off-topic but SAS Viya has image processing capabilities: https://communities.sas.com/t5/SAS-Communities-Library/Image-Processing-with-SAS-Viya/ta-p/568082

And SAS Event Stream Processing has similar capabilities: https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2018/2103-2018.pdf