SAS and Hadoop interactions

[juanvg1972]

Hi,

I would like to know a bit about SAS and Hadoop integration.

I know that you can connect SAS to Haddop via libname using SAS/ACCESS to Hadoop. As far as I know in that case you interact with Hadoop using HiveQL. You can insert SAS tables in HDFS using hiveQL and you can read Hive tables from SAS.

Other way is using data loader product. I think it also connect with HiveQL

Also you can use proc hadoop to execute hdfs commands.

I would like to know how you can do analytics procedures in Hadoop. For example High performance analytics procedures. I have read that this HP procedures can work with Hadoop, but how does it worsks??, is also a connector using HiveQL?, can you execute the procedures in the haddop clusters using map-reduce??, I wnat to know more about thin interaction....if you only gest data from Hadoop or if you can take advantage of hadoop clusters performance executing in a parallel architecture.

Other question... SAS Visual Analytics and Hadoop,...I supopose that you can get information from Hadoop (vía HiveQL) and upload to LASR server..is it right??, all the calculations and aggregations are made in LASR not in Hadoop cluster..is it right??

Thank in advance

[LinusH]

Great that you are curious. But do you have an issue?

If you are interested in starting to use SAS with Hadoop, a contact to your local SAS office is a good start.

For an overall map of SAS-Hadoop integration, Paul Kents Global Forum papers are a good start:

http://support.sas.com/resources/papers/proceedings15/SAS4642-2015.pdf

For HPF, it works like an in-database processing. For the details in the set up, again, talk a SAS representative (also because it's surprisingly hard to get this information from the products page).

For Visuall Analytuics, the standard setup is to load data in hfs, but in a SAS LASR specific format, optimized to quickly load into memory. Perhaps you could use Hive (or in a near future), but wouldn't be optimal since SAS maps its's on data byte-wise to memory on the local server, Hive data is spread around the cluster, so such 1-1 data mapping isn't possible (I think).

[AnnaBrown]

Hi juanvg1972,

To add to LinusH’s response, here are some resources that may help answer your questions.

A couple technical webinars led by SAS Technical Architect Jeff Bailey:

Getting Started with SAS® and Hadoop

Topics include:

• Move files to and from HDFS.
• Analyze data stored in Hadoop with SAS.
• Set up the SAS/ACCESS Interface to Hadoop engine and query with Hive.
• Execute MapReduce from SAS.

SAS^® Integration with Hadoop: Part II

Topics include:

• SAS/ACCESS^®Interface to Hadoop engine support for HiveServer2 plus Kerberos.
• The new SAS/ACCESS Interface to Impala engine.
• Scoring models inside your Hadoop cluster without moving data.
• How the SAS^®LASR ^TM  Analytic Server uses Hadoop data.

Two papers with general overviews of how SAS works with Hadoop:

SAS® and Hadoop Technology Overview

Bringing the Power of SAS to Hadoop