topic Re: SAS integration with Spark in SAS Data Management

SAS integration with Spark

Harishankar_90 — Thu, 26 Oct 2017 05:46:20 GMT

Hai...

I am new to SAS and just started with Base Programming. When Iam finished with it, planning to do DI. My query is whether we can integrate SAS with Apache Spark just like we do it with Hadoop. If so, in what language can we do it ie Scala, Mapreduce etc.

Re: SAS integration with Spark

sandykota — Thu, 26 Oct 2017 05:53:27 GMT

Hi,

I hope the below weblink answer your question:

https://blogs.sas.com/content/sascom/2016/03/18/your-apache-spark-questions-answered-here-and-at-stratahadoop/

Re: SAS integration with Spark

LinusH — Sat, 28 Oct 2017 21:29:22 GMT

Not quite, since data loader for Hadoop is separately licenced.
But if you use, you create "directives" that holds the logic you define. They are in turn possible to call from DI Studio:
http://documentation.sas.com/?docsetId=etlug&docsetTarget=p1fbrvn25x937sn1vtt5ag9epxfh.htm&docsetVersion=4.903&locale=en

You can execute MapReduce tasks from PROC HADOOP (there's a corresponding transformation) but that wouldn't be Spark I guess.

Re: SAS integration with Spark

LinusH — Sat, 28 Oct 2017 21:30:30 GMT

Re: SAS integration with Spark

JBailey — Tue, 13 Feb 2018 21:11:25 GMT

Hi @Harishankar_90

I am the SAS Product Manager for Data Access and In-Database. Right now, I don't have a good answer for you. We have no specific SAS/ACCESS product for Spark. That being said, it is being looked into. That is all I can say at this time. Stay tuned.

It is true that Data Loader for Hadoop does include Spark functionality but I don't think it covers what you are interested in.

Best wishes,

Jeff