Hi,
I am wondering if Spark can be made natively available on SAS Viya? As I don't want to push data outside of the Viya environment.
Many thanks
G
Are you asking if you can deploy a Spark engine inside a SAS Viya deployment?
ASFAIK SAS doesn't provide that.
So the question is what is your real constraint?
What data do you have in CAS and how is it stored?
If you chose to store the data in 3rd party DBMS/engine, you can make that available to your Spark cluster of choice.
What options do you have, and where have you deployed Viya?
In my mind Saprk is not a language, it's a platform or engine for data management, query and analysis, similar to the engines Viya provides (CAS and SAS Compute). So it makes little sense to have a Spark cluster within SAS Viya platform. Again, you can deploy open source Spark and have the same data availble to it as you have in SAS.
Spark supports SQL, Python, Java and Scala, (and perhaps more since it's open source), of wichSAS supports more or less SQL, Python and Java.
What is your requirement?
@gra_in_aus - Here is the documentation link for Viya Spark support. Upskilling on Spark / Databricks would be done on the Spark platform itself. SAS can be used to both query Spark-based data such as Databricks, or to run in-database processes on Spark.
EDIT: Just to be clear Spark / Databricks is a third-party product so it will always sit "outside" of Viya on its own cloud-based platform.
Registration is now open for SAS Innovate 2025 , our biggest and most exciting global event of the year! Join us in Orlando, FL, May 6-9.
Sign up by Dec. 31 to get the 2024 rate of just $495.
Register now!
Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.
Find more tutorials on the SAS Users YouTube channel.