My organization recently purchased Databricks as a complimentary analytic platform to run alongside SAS. While using Databricks natively through their notebook yields awesome performance via Spark technology, I am now wondering if I can use Databricks to replace all my current Oracle connections. Initial tests using pass-thru via ODBC has showed awesome performance gains (queries that would take hours if run in an Oracle database finish in under one to two minutes using a 11 node cluster). In spite of the potential, performance is only good when SAS does not try to orchestrate the completion of a query, and further, expecting users to only use pass-thru when connecting to Databricks is unrealistic. We are also doing a POC for MEMSQL as an alternative solution. My question, Margret or Tony, if you're reading this post please let me know if you've done any testing with these technologies :-), has anyone tried using Databricks or MEMSQL to underlay SAS 9.4 or SAS Viya? We recently purchased Viya and will be deploying in prod in late summer. We need to decide on a data layer, Oracle, Databricks, MEMSQL, or HDFS? So many options and I am hopeful someone has already blazed a path. If not I guess this will be a whitepaper for next year's SASGF! Thanks everyone!
... View more