SAS Viya on Microsoft Azure Marketplace: Accessing your Data on ADLS Gen2

1 Like

SAS Viya has been released in the Microsoft Azure Marketplace last month during SAS Explore virtual user conference. With the click of a button on a pay-as-you-go basis, one can start working on SAS Viya in minutes. Being able to get access so quickly to SAS Viya is a huge step forward.

Now that you have SAS Viya up and running on Microsoft Azure, where do you start? How do you get data in there?

In this post, we will review how to access data from Microsoft Azure Data Lake Storage Gen2 or simply ADLS2.

What is ADLS Gen2?

ADLS Gen2 is a low-cost object storage solution for the cloud, used for building enterprise data lakes on Azure. Microsoft customers use ADLS2 for storing massive amounts of structured/unstructured data.

You are using ADLS Gen2 when you create a Storage Account and check “Enable hierarchical namespace”:

xxx

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Then generally your data is organized across blob containers and sub-folders (Storage Browser view):

What do you need from ADLS2 to setup a connection in SAS Viya?

You need:

An ADLS Gen 2 Storage Account where you have “Contributor” and “Storage Blob Data Contributor” roles
- Name of the Storage Account
- Name of the Blob container
- Path to read data from

An Azure Active Directory Application configured to access ADLS2
- Tenant (or directory) ID
- Application (or client) ID

More details on the ADLS2 configuration is available in this post.

How do you access ADLS2 from SAS Viya?

Here are the different ways to access ADLS2 from SAS Viya:

Capability	Engine	File Type Support	Access Type
ADLS FILENAME engine	SAS Compute Server	Any file type supported by SAS	Read/Write
ORC LIBNAME engine	SAS Compute Server	ORC	Read/Write
ADLS Data Source (CASLIB)	SAS Cloud Analytic Services (CAS)	CSV ORC Parquet	Read/Write

First time

The first time you will try to access an ADLS2 file, you will be asked to authenticate. You will see a message like:

ERROR: To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code EVXXXXXW4 to authenticate.

Open the URL in a separate browser tab, paste the code indicated in the SAS log, log in with your Azure account following the different steps. When you are successfully logged in, re-run the piece of SAS code that threw the authentication error. It should now run successfully.

Examples

Importing a CSV file from ADLS2 into a SAS Data Set:

%let TENANTID=xxxxx ;
%let APPID=xxxxx ;

options azuretenantid="&TENANTID" ;

filename contacts adls "data/contact_list.csv"
	accountname="svomasa"
	applicationid="&APPID"
	filesystem="mydata" ;

proc import file=contacts out=contacts
			dbms=csv replace ;
run ;

Read and create an ORC file from/to ADLS2:

options azuretenantid="&TENANTID" ;

libname orcadls orc "/data/orc"
	storage_account_name="svomasa"
	storage_application_id="&APPID"
	storage_file_system="mydata"
	;

data orcadls.extract_10years(drop=facilityage) ;
	set orcadls.megacorp (keep=facilitystate revenue facilityage where=(facilityage>=10)) ;
run ;

Load a Parquet file from ADLS2 to CAS:

cas mysession sessopts=(azuretenantid="&TENANTID") ;

caslib adls datasource=
   (
      srctype="adls",
      accountname="svomasa",
      filesystem="mydata",
      applicationid="&APPID",
      resource="https://storage.azure.com/", 
      dnssuffix="dfs.core.windows.net"
   ) path="data/parquet/userdata.parquet/" subdirs libref=adls ;

proc casutil ;
	load casdata="userdata1.parquet" casout="userdata1" ;
	list tables ;
quit ;