SAS integration with Spark

[Harishankar_90]

Hai...

I am new to SAS and just started with Base Programming. When Iam finished with it, planning to do DI. My query is whether we can integrate SAS with Apache Spark just like we do it with Hadoop. If so, in what language can we do it ie Scala, Mapreduce etc.

[sandykota]

Hi,

I hope the below weblink answer your question:

https://blogs.sas.com/content/sascom/2016/03/18/your-apache-spark-questions-answered-here-and-at-str...

[LinusH]

Not quite, since data loader for Hadoop is separately licenced.
But if you use, you create "directives" that holds the logic you define. They are in turn possible to call from DI Studio:
http://documentation.sas.com/?docsetId=etlug&docsetTarget=p1fbrvn25x937sn1vtt5ag9epxfh.htm&docsetVer...

You can execute MapReduce tasks from PROC HADOOP (there's a corresponding transformation) but that wouldn't be Spark I guess.

[JBailey · Accepted solution]

Hi @Harishankar_90

I am the SAS Product Manager for Data Access and In-Database. Right now, I don't have a good answer for you. We have no specific SAS/ACCESS product for Spark. That being said, it is being looked into. That is all I can say at this time. Stay tuned.

It is true that Data Loader for Hadoop does include Spark functionality but I don't think it covers what you are interested in.

Best wishes,

Jeff