SAS 9.4 M7 and Azure Delta Lake

[SASWayne]

Hello Experts,

Can SAS 9.4M7 access delta tables from Azure delta lake?

if yes to the above question, could someone help me with steps or documentation to establish SAS and delta lake connectivity please?

Thanks in advance!

[SASKiwi]

I assume you mean Azure Data Lake? AFAIK, ADL is a "souped up" version of Microsoft SQL Server. So most likely you will need either SAS/ACCESS to SQL Server or SAS/ACCESS to ODBC or JDBC. Do you have any of these? Do you want to connect from a server or a PC install of SAS?

[SASWayne]

Good day @SASKiwi - Thanks for responding!

I mean Azure Delta lake....

for reference - https://docs.microsoft.com/en-us/azure/databricks/delta/

and https://delta.io/

We do have SAS/ACCESS to ODBC licensed and trying to set this up on SAS 9.4M7 grid environment (Linux) hosted on Azure.

What I understood about delta lake is that delta tables stored in delta lakes are read and transformed by interfaces such as Azure Databricks, Apache Spark, Azure Synapse etc., I'm wondering if SAS could do the same as these Interfaces.

[SASKiwi]

OK, in theory SAS/ACCESS to ODBC should work. I suspect you need the Databricks ODBC driver which you can download here: https://docs.microsoft.com/en-us/azure/databricks/integrations/bi/jdbc-odbc-bi

You might want to open a Tech Support track to confirm requirements and configuration.

[SASWayne]

Same thoughts here! I was going through the below.
https://databricks.com/blog/2022/03/16/how-to-speed-up-data-flow-between-databricks-and-sas.html

I'll keep this thread open and add the steps here after establishing connectivity. thank you for looking into this!

[shoin]

Interesting. I was wondering if Azure Synapse is could be connected from SAS using SAS ACCESS/SQL SERVER. I will give it a try and post what I found.

-S

[SASWayne · Accepted solution]

We were able to achieve this using Databricks through ODBC. Also, there was hint from SAS TS that support to read Parquet files might be available from SAS 9.4M8. If it happens, hopefully we should be able to directly access Delta lake.

Steps,

1. Installed ODBC Driver Manager

2. Installed ODBC Driver for Databricks.

3. Added DSN information into odbc.ini

4. Added ODBCINI env variable into <sashome>/SASFoundation/9.4/bin/sasenv_local

5. Validated the connection using isql command.

6. Setup DB connectivity at SAS Metadata and other stuff and tested connectivity from SAS clients.