topic SAS 9.4 M7 and Azure Delta Lake in Administration and Deployment

SAS 9.4 M7 and Azure Delta Lake

SASWayne — Mon, 04 Jul 2022 04:50:32 GMT

Hello Experts,

Can SAS 9.4M7 access delta tables from Azure delta lake?

if yes to the above question, could someone help me with steps or documentation to establish SAS and delta lake connectivity please?

Thanks in advance!

Re: SAS 9.4 M7 and Azure Delta Lake

SASKiwi — Mon, 04 Jul 2022 06:45:17 GMT

I assume you mean Azure Data Lake? AFAIK, ADL is a "souped up" version of Microsoft SQL Server. So most likely you will need either SAS/ACCESS to SQL Server or SAS/ACCESS to ODBC or JDBC. Do you have any of these? Do you want to connect from a server or a PC install of SAS?

Re: SAS 9.4 M7 and Azure Delta Lake

SASWayne — Mon, 04 Jul 2022 06:58:44 GMT

Good day @SASKiwi - Thanks for responding!

I mean Azure Delta lake....

for reference - https://docs.microsoft.com/en-us/azure/databricks/delta/

and https://delta.io/

We do have SAS/ACCESS to ODBC licensed and trying to set this up on SAS 9.4M7 grid environment (Linux) hosted on Azure.

What I understood about delta lake is that delta tables stored in delta lakes are read and transformed by interfaces such as Azure Databricks, Apache Spark, Azure Synapse etc., I'm wondering if SAS could do the same as these Interfaces.

Re: SAS 9.4 M7 and Azure Delta Lake

SASKiwi — Mon, 04 Jul 2022 20:10:36 GMT

OK, in theory SAS/ACCESS to ODBC should work. I suspect you need the Databricks ODBC driver which you can download here: https://docs.microsoft.com/en-us/azure/databricks/integrations/bi/jdbc-odbc-bi

You might want to open a Tech Support track to confirm requirements and configuration.

Re: SAS 9.4 M7 and Azure Delta Lake

SASWayne — Tue, 05 Jul 2022 02:11:15 GMT

Same thoughts here! I was going through the below.
https://databricks.com/blog/2022/03/16/how-to-speed-up-data-flow-between-databricks-and-sas.html

I'll keep this thread open and add the steps here after establishing connectivity. thank you for looking into this!

Re: SAS 9.4 M7 and Azure Delta Lake

shoin — Wed, 20 Jul 2022 22:07:48 GMT

Interesting. I was wondering if Azure Synapse is could be connected from SAS using SAS ACCESS/SQL SERVER. I will give it a try and post what I found.

-S

Re: SAS 9.4 M7 and Azure Delta Lake

SASWayne — Fri, 19 Aug 2022 13:44:42 GMT

We were able to achieve this using Databricks through ODBC. Also, there was hint from SAS TS that support to read Parquet files might be available from SAS 9.4M8. If it happens, hopefully we should be able to directly access Delta lake.

Steps,

1. Installed ODBC Driver Manager

2. Installed ODBC Driver for Databricks.

3. Added DSN information into odbc.ini

4. Added ODBCINI env variable into <sashome>/SASFoundation/9.4/bin/sasenv_local

5. Validated the connection using isql command.

6. Setup DB connectivity at SAS Metadata and other stuff and tested connectivity from SAS clients.