8 Best practices for moving analytics to the cloud

[#1]

By James Ochiai-Brown and Mathias Coopmans

More than half of enterprise workloads and data are expected to be in a public cloud by mid-2021. That’s one of the key findings of the Flexera 2020 State of the Cloud Report according to TDWI. Cloud adoption has accelerated with COVID-19 and where data goes, analytics follows.

Over the last few years, SAS has worked with many organisations in moving analytics to the cloud. We are now at a point where we can identify some best practices and we will share 8 of them with you here. These are also explored in a recent webinar.

1. Get a clear view of your cloud strategy – and align

Firstly you need to understand your organisation’s cloud strategy and how to align with it. That’s more than just what cloud vendor you are going to go with. It should also address the following.

Expected benefits of moving to cloud
Cost reduction is sometimes cited as the main benefit, but other aspects may be more significant. Subscription pricing for cloud services enables costs to be shifted from capex to opex. Environments can be deployed much faster and scaled up to meet demand. You need to know which of these are most important so your design for analytics on cloud can provide these benefits.

Cloud data strategy

What data will be stored on cloud? What data storage technologies will be used? How will data be managed and shared? Analytics is both dependent on data and creates data so it essential to know how analytics will fit into the cloud data landscape.

XaaS strategy

Cloud offers Infrastructure as a Service (IaaS) which will enable you to move your on-premises analytics to virtual infrastructure in the cloud. If you don’t want the overhead of managing your analytical platform you can have it hosted and managed for you with the Platform as a Service (PaaS) approach. Perhaps your business users simply want to access an application with Software as a Service (SaaS). You need to understand where your organisation is leaning and which approach has been deemed suitable for each type of service.

Constraints

What are the security policies and governance related to cloud? If you don’t know these you may run into significant obstacles and delays.

Roadmap

What’s the overall timeline for moving to the cloud? What’s driving it? If you know that infrastructure is nearing end of life or the data centre contract is up for renewal it will give context to your plans.

2. Assess your current state

Before you move to cloud it’s essential to understand the current state of your analytics platform. A comprehensive assessment exercise will tell you the range of processes and data on the platform and the business processes that rely on analytics. Automated scripts should be run to identify any code or functionality that may prove challenging to migrate e.g. commands related to Windows operating system or desktop tools. Data volumes and formats need to be assessed as it may take weeks or months to convert and migrate all data to the cloud.

If you are considering a migration from SAS 9 then it’s best to use the SAS Content Assessment toolkit. This will examine the characteristics of your SAS 9 system and highlight issues that may present challenges. SAS Consulting offers services to assess cloud readiness, including the use of the SAS Viya readiness assessment utility to determine how SAS code can be converted to leverage CAS.

Additionally, be careful of what you don’t know and can’t see. Your organisation may have many more analytics processes then you are aware of. They may be undocumented, totally user driven, developed in a hidden corner and reliant on unsupported tools, yet they could be extremely valuable for the organisation with many business dependencies.

3. Use the cloud for experimentation

How fast can your organisation innovate with analytics? Do you want to try out and prove an idea before you commit to it?

Conventional on-premises technology deployments typically take weeks or months and require up-front investment, even for a proof-of-concept. A pay per use model on cloud asks for very limited commitment to get started. You can quickly find out which ideas work and which don’t. The value is tangible and can be put into a business case to obtain funding for further development and roll-out.

For example, we have worked with a large insurer that is on a digital transformation journey. Their board fully understands the role and potential value of analytics on this path. All initiatives and experiments with data and analytics are heavily encouraged. As many of these initiatives are merely tests, highly agile environments are needed while they are working out what their direction is. They are constantly putting up sandboxes in the cloud where their innovation centre can freely experiment with data and help prioritise future projects.

4. Select the right migration approach according to your priorities

You won’t be moving everything to cloud at the same time or in the same way. Much depends on the current state, your business drivers and the target state vision as described in your cloud strategy. You will need to consider the various analytical applications and assess which kind of migration will be most appropriate. Then, prioritise the move of each one according to the urgency, cost savings, opportunities and dependencies.

In 2011 Gartner outlined 5 ways to migrate applications to the cloud: Rehost, Refactor, Revise, Rebuild and Replace. Since then, a few variations have been suggested with alternative Rs: Re-platform, Re-architect, Retire etc. Whichever set you work with, it’s important to know your options and the pros and cons of each.

Rehosting, otherwise known as lift-and-shift, may be appropriate for legacy applications that still function reasonably well but are running on expensive or outdated infrastructure. Perhaps the existing infrastructure is out of support or there is a plan to close the data centre. Rehosting on cloud may be the easiest way to extend the life of the application and benefit from some lower infrastructure costs.

However, the benefits of rehosting are limited as the application will not be designed to take advantage of cloud features. For more significant cost reduction or performance gains consider refactoring or re-architecting your application to use cloud native services. Take the example of Cox Automotive, an organisation heavily reliant on analytics that had struggled with performance and uptime issues. They migrated to a re-architected analytical platform on AWS with serverless functions (Lambda), elastic load balancing and auto scaling. As a result, they reduced operational costs, improved performance, maintained uptime and now have the flexibility to deploy new services to support business growth.

5. Put analytics wherever the data is

Data Gravity as a concept means that when data is placed in the cloud it will attract other data, services and analytical workloads. For efficiency and performance, you need to locate your analytics near to your data. Typically, this will involve putting analytical workloads on one or more cloud platforms, on premises and even on edge devices such as manufacturing plant and vehicles.

Analytics is a matter of people and skills as well as technology. Consider the years of experience of those analysts who work with data on premises. Once data moves to the cloud those people should be able to continue their work against the new data sources using the appropriate cloud-based analytical tools.

6. Utilise the power of the cloud to scale

Cloud offers you the possibility of scaling, both vertically and horizontally.

Vertical scaling involves increasing the size of the machines. You can do this easily with cloud IaaS, for example upgrading a 16 vCPUs machine to 32 vCPUs in just a few minutes. It’s simple but, of course, you can only scale up as far as the largest machine type available.

For more extensive scaling you need to expand horizontally, adding additional machines. The SAS Viya massively parallel processing (MPP) configuration for the CAS in-memory analytics engine uses a cluster of machines and allows you to add worker nodes to increase processing power and memory. With cloud IaaS these worker nodes can be provisioned quickly without the typical delays experienced with on-premises deployments.

Time-series forecasting is particularly suited to parallel processing over a cluster of machines. Consider a forecast on 1.5 million time series. Running on a single processor, this would take around 130 hours but on a 400 vCPUs distributed CAS you can run this in around 5 minutes! Cloud IaaS allows you to provision a cluster of this size when you need it and you can release the machines once you are done so you are only charged for the time you use them.

Scaling gets even easier on cloud when you adopt cloud native technologies based on microservices, containers and container orchestration. Instead of provisioning VMs and installing software you deploy modular software in containers on a cluster of machines. The container orchestration system, Kubernetes, determines where the containers will run and makes copies of containers to scale automatically in response to increasing demand. The latest release of SAS Viya is built in this way.

7. Use multiple clouds depending on your purpose

A range of clouds is available, from the well-known public cloud vendors to smaller, local players, and software-as-a-service offerings. SAS Cloud, powered by Microsoft Azure, combines the power of major cloud provider with managed services specifically for SAS analytical workloads.

IDC declares 2021 as the year of multi cloud and Gartner also sees multi-cloud as a key trend, reducing vendor lock-in and providing flexibility in procurement, functionality and risk mitigation. According to the Flexera 2020 State of the Cloud Report 93 percent of enterprises have a multi-cloud strategy, either with multiple public clouds or a combination of public and private clouds.

Organisations with global operations may want to select a cloud provider for each region to meet local requirements for data governance, performance or support. One cloud vendor may offer a service or set of capabilities that meets the needs of a particular business area whereas another vendor may have broader capabilities to serve the rest of the organization. Private cloud is likely to offer better performance and easier integration when working with data sources in the same on-premises data centre.

Working with data across these clouds may be a challenge but increasing use is being made of APIs to call services and exchange data. For example, SAS offers a Data Connector to Salesforce and a SAS/ACCESS interface to Google BigQuery.

8. Enable self-service analytics

Prior to cloud, if a business area needed some new analytical capability the IT department would assess their needs and propose a solution. There would be a delay while the business case was approved then further delays for procurement and provisioning.

Cloud enables faster provisioning and can put control in the hands of the business. Compute resources are available on demand and it’s easy to deploy analytical software either through a quick-start script or using a framework for spinning up instances when required. Pay-as-you-go billing allows each business unit to purchase computing resources itself without going through a central procurement function. With software-as-a-service the business can cut IT out altogether.

As IT is no longer the provider of analytics technology it has to redefine itself as enabler, giving business units the ability to self-serve their analytics needs. At the same time, it must establish the strategy and governance to ensure efficiency, interoperability, data quality and security. Business units can then operate within this framework with the support and guidance of IT.

SAS is committed to offering software-as-a-service, giving users to access the software when they need it. The latest release of SAS Viya running on Kubernetes allows us to move further in this direction. It offers the prospect of self-service analytics within an organisation with users able to select an analytical software environment through a portal and have it provisioned automatically, enabling them to derive the insights and support the decisions the business needs.